"So why am I trying things this way?"

"Because, if this just works, it will be faster."

"And I am not sure that this won't work.  I haven't tried.  I haven't checked."

"So we're experimenting.  We are looking.  We are going forth and finding out."

"When to that underlying emotional spirit of curiosity, questioning, 'blah blah blah'(*), and glorious discovery, is added the inspiration of Law, it becomes what dath ilani call 'blah blah blah'(**)..."

"Right, uh, let me just pause and tap everyone with Communal Shared Language (Baseline)."

Keltham then goes about from person to person.  He's glad he got two of those spells, today; with this many people, he can only give most researchers and candidates an hour of Baseline apiece per casting.

(*)  'Maniacal-experimentation'.
(**)  'SCIENCE!', lit 'Civilization's-learning', with 'learning' as in a child's learning.  Distinguished from ordinary everyday systematic experimentation using math, in that it is a societywide effort to COMPREHEND and CONQUER the ENTIRE UNIVERSE.

So the entity - Keltham, Keltham, his name is Keltham and he goes by it, it's going to sound so weird if she says "the entity" out loud, even though it probably won't actually cause any wall problems - Keltham actively wants dumb questions, even though the project leaders obviously don't. That's good. It should be easier to think of questions that don't reveal anything if she doesn't have to also filter for being incredibly insightful. And hey, maybe she won't even have to contribute much to classroom chatter in the first place; everyone else is probably just as capable of asking dumb questions as she is. Not that she's going to make any ill-founded yet confident assumptions about what other people are thinking when she can't see it, because she's decided that there's quite enough of that around here, and she's going to at least find less repetitive ways of being constantly wrong.

(Also... apparently Keltham already knows that the way they're mostly taught involves remaining silent, so why do they have to go to so much effort to act like it isn't? But that's a question from her real self, not her alter-world self; she tucks it away and resolves to stick with what the project leaders have told her, because they've obviously had more time to think about the picture they're presenting to him.)

Willa Shilira is sitting in the back of the class and she's very unhappy about it, but you couldn't tell that by looking at her. She has less Chelish dignity and stonefaced composure in her than perhaps any of the other students, but that's still plenty to be unassuming in a crowd of almost two dozen. (She's mentally acute enough to count the twenty-three visible people in the room while barely thinking about it, but not perceptive enough to even notice Broom being the twenty-fourth.)

Willa came into the project bright eyed and excited about Law and learning and superpowers, desperate to be special. She was so desperate for it that when a chance to ask questions came up, she tried to ask one mostly just to show off:

"Of course if the project is so important we should all be getting intelligence headbands, right, and I can put that impertinent demand in fancy probability language to prove I understood the homework too! I didn't stop to think that economic reasons were a trivial concern and that the real thing I should be worried about was fooling Keltham, even though you just gave a giant speech about how the important thing was fooling Keltham. So obviously I'm a complete idiot and you should never trust me with anything at all!"

That's not actually what she said, but it might as well have been. They didn't torture her for it at all which was super weird, but now alter Willa is 'studious, shy, and hesitant.' Forever. And she has to mentally run all her questions by Asmodia now, of course, to make sure they aren't suicidally stupid to say out loud. And she can guess that means if she ever actually has a good one it'll get given to someone else who isn't supposed to be 'shy and hesitant.'

She could choke herself with a rusty spoon.

She messed up and she knows it and she's mostly just sulking, angry at herself. There's a simmering annoyance with Asmodia for forcing alter Willa on her, and still a lingering resentment towards Sevar for capturing most of her soul-profits, but in the end the reason she can't be a model student actor agent is that she's too obsessed with being special and not obsessed enough with winning. It's a terrible failing in her and it's worse that she knows deep down that it's still true, it's not something she knows how to change.

She's not going to be able to distinguish herself to Keltham by asking clever questions, and honestly even if she was she maybe couldn't trust herself to do it safely. Instead she'll have to distinguish herself to Asmodia by thinking useful thoughts. Then maybe eventually Asmodia won't hate her anymore she'll be out of the doghouse and at less risk for washing out entirely.

Having tapped everyone with Share Language - including Broom, once Broom reminded Keltham of his existence - Keltham speaks forth again:

"The first thing to do in this experiment... is something I expect you'll all find way too natural, namely, don't say anything and control your facial expressions.  That's not the usual rule in this classroom, but in this case I want you to confront a problem separately and without leaking information to each other, so no comments out loud on what I'm about to do or say here.  If you want to say something to me, or have a question, say 'Message' and then use Message."

Keltham goes to the wall, casts Prestidigitation, and draws three Baseline numbers via color-changing the wall's surface.  Possibly using Prestidigitation all the time will help train magic use to where it becomes more innate and reflexive for him?  Though he should actually check that concept with Carissa at some point, because it's a time-costly experiment.

The Baseline numbers that Keltham draws are 2, 4, 6.

(It should be immediately apparent that Baseline numbers are both easier to draw, and more visually distinguishable, than Taldane digits.  Just one more little reminder that almost everything about dath ilan was designed and not just allowed to happen.)

That is not, actually, immediately apparent to Korva; maybe it would be if she were focused on it, but right now she's busy trying to control her heartbeat even though she's pretty sure nobody can hear it leaking evidence that she's freaking out a little about whatever bizarre new kind of math this is going to be.

"I am now the - world? Nature? - 'environment' in Baseline," in case any of the concepts of 'thing that interacts with the agent and is separated from it only by a false conceptual boundary' make it across in translation.

"On each round, you input to me a sequence of any three integers, and I output to you either 'yes' or 'no'."

"You are learning me, unraveling me, theorizing about me and experimenting on me; you are trying to predict what makes me say 'yes' or 'no' on each round.  To be clear, I wouldn't particularly suggest that you think of these symbols 'yes' or 'no' as really meaning anything, except insofar as you think you've determined experimentally what makes me say them."

"You will now write down 2, 4, 6 on your paper - Taldane is fine, I can still read that.  When you're done, raise your hand, and I'll mark down on your paper whether this initial input causes me to output 'yes' or 'no' to you."

"If you think you've guessed the Law that governs my outputs, you can write that down, and I'll write 'correct' or 'wrong' next to that, where those words do have their standard meanings."

Willa's still a little jumpy.

When he first asks them to raise their hands just for the '2, 4, 6' she suspects it's a trap, and writes just slowly enough to make sure she sees a couple other students raise their hands first. Turns out it isn't a trap?

2, 4, 6: YES

8, 10, 12: YES

Willa would ordinarily guess here, right away, if she hadn't just been admonished to be careful. But she was, so she stopped, thought for a second, and then-

-6, -8, -10: YES

3, 1, -1: YES

The law is that the absolute value of the number must cha- Willa crosses that one out, realizing something.

5, 7, 5: YES

And there's a triumphant expression that's too subtle for Keltham to see but not too subtle for any Chelish students that might happen to be looking at her.

The law is that the the absolute value of the change of the number must be 2 each time.: WRONG

She experiences existential anguish, and then pauses in terror, expecting Asmodia to scream at her in indignation.

Asmodia does in fact yell at her, on relay through Security:  Control your facial expressions and don't fucking mess up Keltham's test, you fucking moron.  He may be able to tell from the results he gets if we didn't all follow instructions.  Luckily I don't think anybody who wasn't Security was stupid enough to be looking at you.

Don't do anything alterWilla isn't doing, FUCKING PERIOD.  If you prove unable to fucking understand that, alterWilla will fail to keep up and sadly resign from this Project.

Of course she messed up again, apparently she can't do anything right. She fiercely pushes everything down, and at least for a little while, her expression will remain thoroughly blank.

Idly translating Keltham's pre-Share Language sentences, Alexandre Esquerra, second-circle wizard and new Project Lawful researcher, discovers that Baseline has a word for "a societywide effort to COMPREHEND and CONQUER the ENTIRE UNIVERSE," and Alexandre Esquerra has never been so happy in his life. 

Then, Keltham's test. Alexandre will, of course, do better than his rivals; his first instinct is that he needs to solve it in as few guesses as possible, which he immediately crushes, stamps down on. He should not win the contest, he should learn, getting as much information as he can from Keltham before he dares hazard any guess. Once he's solved it, he can dawdle around and do extra checks and answer everything.

2,4,6: YES.

All right, think of this as spellcraft, to make sure magic items or spells stabilize; a ratio of 2,4,6 is in balance, is 2,4,7?

YES. 1,4,6? YES. 2,3,6? YES.

So much for the 'ratios' idea. That's thoroughly disproved. Well, Keltham's knowledge is strange and alien, and he should not expect it to match that of his own civilization. 6,4,2? YES.

... All right, so even reversing the order works; he's starting to get annoyed, he doesn't have good alternate possibilities and so he's stabbing in the dark. He isn't letting it show, of course, he's not a child. Does 1,1,1 stabilize? YES...

There has to be some spell structure that doesn't stabilize under these absurd laws, doesn't there? How about if he tries to break the bounds with -6000,7000,0: YES. Smoldering annoyance. Desire to burn everything. He can manage, while hating, if he could not manage he would not be alive. 500,0,500? YES. 100000000000,0,0. YES.

All right, he knew that was going to happen, his actual expectation was that one hundred billion would stabilize, which, per Keltham's rules, means he already knew it. The actual goddamn law is 'any three numbers', isn't it? There has to be something that isn't accepted unless Keltham is a -

Of course. Yes. Keltham is a sadist. He should not have considered this question as 'the output of a superior civilization bent on teaching its students', he should have considered the question as 'the output of a sadistic teacher intent on inflicting suffering on his class'. Hell is the destruction of hope, such as his hope of successful learning. He'd like to let security know he thinks he has a correct guess, but wants to make sure he isn't the first one to write it down.

(And, while he does that, he'll continue writing random strings of numbers so Keltham doesn't guess. 313-496-386? YES, of course.)

Security relay from Asmodia:  Stop.  Stop trying to act in ways that your alterCheliax self wouldn't.  Just stop.  Don't do anything fancy.  Just.  Be.  AlterAlexandre.  He'd go ahead and guess, so go ahead and guess before Keltham notices you're acting weird.

(...is this what it's like for Aspexia Rugatonn when she tries to explain to anyone about what probably seems to her like an incredibly simple and straightforward concept of just following orders?  It totally is, isn't it.  No, Asmodia isn't feeling even slightly sympathetic, because Aspexia Rugatonn.)

... All right, yes, acknowledged, he rather deserved that. The law is that any sequence of three integers will get YES.

WRONG.

This sounds even worse than normal math!! 

She writes down 2, 4, 6 on her paper in Taldane, as instructed. She raises her hand, being careful not to be the first one.

2, 4, 6: YES.

1, 1, 1: YES.

Huh. Okay, different numbers, all odd, same pattern. 

3, 5, 7: YES.

Opposite order.

7, 5, 3: YES.

Uh, branch out more, so you can see some no’s. 

7, 8, 9: YES.

Her heart is hammering out of its chest, and it shouldn’t be. She’s so scared of being the last one to figure this out. Realistically, not everyone will, but they can kick as many people off the project as they need to, so -

1, 2, 3? No - 1, 9, 2, that’s more different. 

1, 9, 2: YES.

Nothing to be gained from not guessing, probably? Guess the obvious. 

Guess: Always outputs “YES”.  And she adds another number guess, so as not to waste Keltham’s time when her first obviously-wrong-guess is wrong.

2, 5, 1: YES. WRONG.

Maybe it can be any difference?

101, 102, 101: YES

... or maybe it can just be anything at all?

101, 102, 100: YES

Let's just make sure.

1, 10, 107:

This time, Keltham fails to notice her for a round, because she's hiding in the back like a mouse and she hates alter Willa. Eventually though he notices her again.

1, 10, 107:YES

Let's be really sure.

pi, 5.67, 78/7: INVALID INPUT

And now she remembers they were supposed to be integers. She needs to get it together. It's probably wording and she's just phrasing it wrong, it has to be.

The law is that the three numbers must be integers.:WRONG

The law is that nature always says yes.:WRONG

Maybe if she throws enough things at the wall he will eventually tell her no.

-5, 10, -7:YES

1 is a pretty special number.

1, 1, 1:YES

Zero is too.

0, 1, 2:YES

Maybe the numbers just need to be really large.

10^20, 10^20, 2:YES

Is it really just a tricky wording after all?

The law is that if you give 3 valid inputs nature always says YES.: WRONG

No. YES to everything, absolutely everything, except something she can't think of. She's just stupid, obviously.

...people sure are making some assumptions that dath ilani wouldn't make!  Possibly human beings just naturally make a lot of assumptions, and naturally stay in small mental boxes, that dath ilani kids have already been implicitly jailbroken from by the time they first encounter this problem?

Alexandre will burn this useless scrap of paper! (No, he will write -3, -6, -9 and get back a YES.)

... alterAlexandre would snap even alterAlexandre is not weak. 0,0,0. YES. He continues his search for NO in the endless sea of YES; 2,4,8? (Yes.) 3,9,27? (Yes.) 8,4,2? (Yes.) -2,4,-8? (Yes.)

Mindful of Asmodia's warning not to do anything alterAlexandre wouldn't, he begins letting some strain show on his face. It is rather less than he's feeling, but he's never been outside Cheliax and doesn't know how much Taldorians conceal their facial expressions. He's used to having some idea of where to search.

Next step: 300, 400, 500. Is there some band where the usual ones don't apply? (YES, that is, SCREW YOU.)

4444,3333,2222... (YES.) wait, Baseline sometimes uses Base-12. 12-144-1728? (YES.) So really, Yes and No are inverted, and he can't find any point where he can stabilize this. 59326-78442-19848? YES. 16,4,2? YES.

... Hmm. Alexandre is starting to notice that Keltham is taking longer and longer to answer.

... ... Let's think about this logically. He's tried ascending order and descending order and powers and in the opposite order. He's been told to only try integers or he'd move into fractions. He's tried to break it from the top and from the bottom and input three of the same number and...
 
 ... Wait. Hmm. Odd. 2,2,4.

(YES, but with a long delay.)

That's... odd. 2-4-2. (YES)
 
Probably someone else has stumbled on the secret, and whatever the secret is involves Keltham having to do a lot of complicated mental math.

So he's trying prime factors, and seeing if there's a slot that the 4 should go in. 4-2-2? (YES.)

... 4-2-2 took less time than 2-2-4. Keltham may have just been distracted, or be waiting pauses of ambiguous length. 3-3-3? YES. Yes, yes, as expected, but if it was 'all primes and we don't count 1 as a prime' he doesn't know how he could have lived with himself. (In a dry, slightly ironic tone.) He hesitates a bit to do some quick arithmetic of his own, then: 79-11-879?

An immediate Yes.

Well, that should be interesting.

2, 4, 8. Those numbers at least have a relationship.

2, 4, 8: YES.

Did he say the integers have to be less than ten? He didn’t, did he? Okay. 

1, 247, 6: YES.

Okay, try negatives. 

1, -1, 0: YES.

Try all negatives?? 

-5, -247, -8: YES.

Try all zeroes. 

0, 0, 0: YES.

This exercise is STUPID.

5, 3, 15: YES.

Stupid and AWFUL. 

Okay, what other traits do integers have. They can be even or odd. They can be positive or negative. They can be factors of one another. They can be… addition problems, the first one was an addition problem. They can be division problems. 

Try writing down STUPIDLY LARGE numbers, even though this won’t get her anywhere. 

568, 9840348, 653203: YES.

Obviously.

She can’t think of anything but she’s obviously not allowed to give up, even though her heart is dropping like a rock. She gets a sense of curious worry from the butterfly familiar still in her room, over the empathic link, presumably from all of the intense emotions she’s having. She tries to tamp down her current emotions enough to send a coherent message back. The feeling of seeing a warm meal when hungry, the feeling of being chained up alone in a dark room, the feeling of seeing a warm meal again. This is supposed to indicate to Pretender that she’s fine, actually. She’s not entirely sure that she is.

……..okay, wait. When Korva had reason to design a code with things sort of like digits, she did - alternating differently-valenced things? Try alternating things that are different. 

1, 2, 1: YES. 

-1, 2, -3. YES.

 This is obviously doomed, but be thorough anyway. 

1, -4, 2: YES.

She hates this. They’re going to send her to hell right away, and she won’t even have learned anything, and she will therefore be starting at a disadvantage relative to everyone else in hell because she died young and didn’t even get to learn the things that mortals are supposed to learn, so she’ll be worthless for her entire existence and have a boring and fundamentally useless eternity, not even because she was inherently worthless, but because someone sent her into this stupid, ridiculous class for how to be an axiomite made of math just because she knows how HISTORY works.

These aren’t things that alter-Korva would be thinking except that OH WAIT YES THEY QUITE POSSIBLY ARE, GIVEN THAT EVEN IN ALTER CHELIAX SHE ALSO NEVER GETS TO GO HOME UNTIL THE PROJECT IS OVER.

1, 2, 3, 4, she writes down, angrily, in case disobeying instructions is the POINT.

1, 2, 3, 4: INVALID INPUT.

Okay. Not that.

That technically breaks the “always outputs yes” rule. 

3, 2, 1, she writes, and a guess. Guess: Always outputs "YES" when given a valid input sequence.

3, 2, 1: YES. WRONG.

Of course.

What other things can you do with integers. Division problems, sure, that won’t fucking be it but she can test it. 

…wait. 

Maybe she'll just try prime numbers. It's something, at least.

2, 3, 5: YES

It seems to take Keltham a while to answer this time. Maybe it really is primes, somehow?

23, 19, 41: YES

And it's faster this time. Her primes have betrayed her! Maybe the numbers just need to be pretty small somehow, in some confusing way.

2, 3, 6: YES

But it's slow again! Somehow she has something. Maybe.

Sense Motive check, is Keltham:

A) just messing with her, or
B) legitimately having trouble computing this quickly, or 
C) is he wondering why she's so stupid?

Also, Willa's trying to very subtly look around the room at this point to see if everyone else is finishing and she's alone in her pathetic stupidity.

Results:  Keltham is patient but wondering - more like he's wondering about everyone, not wondering about her in particular, his facial expression doesn't change much as he moves away from her.  Possibly he wasn't expecting this problem to be as hard for everyone as it seems to be?

Some frustration here and there is leaking through other students' expressions in a way that Willa can detect, despite Keltham's instruction.

A number of the other previous hires look finished, but such is only to be expected.  Oh, and what's-her-name - Korva - seems to be finished too.

(The Chosen of Asmodeus finished almost immediately, of course, but that was hardly worth noting.)

The more senior researchers don't shake her much. They've had more time. It's only fair.

But Korva's just as new as Willa, and she's already better. It's a dagger in her heart, but she has to keep going. Whatever happens she can't let herself be last. It'd be the end of the world.

Ok. Go time.

Is she absolutely sure it's not a wording thing?

Nature's output to Willa is YES when she writes three valid inputs.:WRONG

She's sure enough for now. Let's try other small numbers, that won't look stupid to Keltham at all, no siree.

3, 4, 6: YES

It takes him a while again, and she has a terrible, horrible suspicion now.

879-97-11? Immediate YES. -1,1,-1. Slow YES. ... Oh? -1,1,2? 1,-1,-1. Both swift YES. Very interesting. 1,-1,1? YES, very slowly. 

What on Golarion could this possibly be? Now he really wants to know. 1,1,-1. Another slow YES. -1,-1,1 YES, slowly again.

... Hmm. He hypothesizes that -1,-1,-1 is quick? He'll test that hypothesis. YES but a slow yes.

Fascinating. He really just wants to know the answer... Wait. -3,0,0? YES but a slow YES.

... He's imagining that Keltham has to do a lot of mental math here because he's trying to calculate - in his model, everything is either positive once Keltham has done some calculations, and that gets YES, or negative, and that gets NO, and that suggests to him that's multiplication of some sort but the different places meaning different things but he isn't sure which place means which...

-2,1,-2?

Keltham gazes at the paper, eyes flickering a bit, and then writes YES.  It took him about as much time as before; relatively slow.

- Wait, actually, no, there's an even dumber test he should do: -2,1,-2

An immediate NO.

... This cannot be that simple. 2,4,6?

NO.

... Alexandre now sees his fault. He had been, fundamentally, assuming that these were standard tests, independent tests, each question isolated from the last. He had not realized... the meaning of what Keltham had said. He had heard the word, and not thought about it, so only now, does he see the truth: That he cannot consider all his steps in isolation. He does not know the Law, yet, but he is one step closer to finding it.

If he was not in Cheliax, he would be smiling.

Among the first things one does with unfamiliar measuring instruments, in standard tests, is check that equipment's reliability-width by measuring the same thing multiple times!  Any child would know that!

You cannot simply assume that, just because you have cast a spell, it is gone. That spells usually do not interfere with other spells is a generalization, not close to a law.

2,4,6 again? Still NO. 3,5,7?

YES, quite slowly, as Keltham's eyes flicker over the whole previous sheet of paper while successfully making it less than completely obvious what he's doing, as might be obvious if his eyes moved slowly down the page in order.

Dath ilani play enough games where direction of eye gaze can be a giveaway that Keltham has at least some training in that particular aspect of Bluff.

... Well, this is still going to be a difficult challenge. He has the hint, but he needs the Law; whatever it is can't be simple addition, because at one point he tossed a hundred billion in. Each slot may still mean something different, and he has such a tiny, frustrating window to tell useful things from his information!

Still, things may be simple. 3,5,7 again?

(NO, instantly.)

3,5,7 again continues to be no? (NO, instantly.)

Excellent. Then he's going to check this again just because Keltham is a sadist. 3,5,7? (NO.)

Then... is it really just that simple? He... he thought it would be much more complicated than this. His guess, this time, is that anything you've tried before gets a 'no' and anything else gets a 'yes'.

CORRECT.

... He cannot believe it took him this long to get it, but yes, he is feeling... joy, at finally mastering a difficult art, at taking an absurdly complicated thing-in-the-world, and studying it, and reducing it down to something so simple, that he couldn't believe it took him so long to understand it. Through pain, to strength. Great is Asmodeus, Lord of All.

(And Alexandre will let a tiny bit of the happiness he is feeling leak through, since alterAlexandre would not hide his expressions as well as trueAlexandre. Just a little.)

Keltham looks happy about it too, in fact! He's trying to hide it, but not at all well by Chelish standards.

...as for that tiny bit of leaked happiness from Alexandre, there's no chance in the Abyss of Keltham getting that.

All of her sequences so far are new. So -

2, 4, 6: NO.

Incredible.

She would have gotten this faster if she hadn’t been thinking about it as a kind of fucking math. Or if she had been a five-year-old. Or if she had been dumber.

Well, she can probably learn to be dumb sometimes, if that’s what it takes.

3, 5, 7. Guess: Outputs "YES" when given a new sequence; outputs "NO" when given a sequence already used.

NO. CORRECT.

She doesn’t collapse in panicked relief on her desk, because she’s not actually five and also that would be disobeying instructions about leaking information. She stares ahead, outwardly calm - which is probably in itself leaking information, because she probably unintentionally leaked a bunch of distress everywhere right around 1, 2, 3, 4. In any case, her soul is crying out in relief, and she gets a sympathetic mental pat from Pretender about it.

Only then does she look around, and see that all of the other new students are still writing. Some of the old researchers - including, of course, the Chosen - have stopped writing, but not all of them.

She really doesn’t know how to feel about this.

Previously, on Willa (the Complete Disaster at Everything):

With a heavy heart, she writes the fateful numbers.

3, 4, 6: NO

Oh no. Well, as bad as this is, she has to be sure.

2, 3, 6: NO

It's a little slow again this time, because of course it is. He had to read up her sheet. But she isn't sure ENOUGH.

1, 1, 1: NO

And she has to make sure she isn't missing something else either. Order could matter too. She is not going to get this WRONG. Not AGAIN.

6, 3, 2: YES

Well fine then. She notices that she still has an internal flinch expecting that WRONG gets written after her answer, in spite of all her reason insisting it won't be.

The law is that if the three numbers have been entered before in order by Willa, nature says NO. If this is the first time the three numbers have been entered by Willa, nature says YES.: CORRECT

She's a massive failure, but she's still relieved. None of it shows on her face though.

2, 4, 6: NO

2, 2, 2: NO

0, 1, 2: NO

900, 600, 300: NO

-1, 0, 1: NO

a, b, c: INVALID

Well, that's some obvious theories that don't seem to be on the right track at all. Back to basics, then: does the rule output the same judgment every time you give it the same input? 

2, 4, 6: YES

- and there's an obvious theory (any set is allowed the second time, but not the first) which does seem to be on the right track, but she's met Keltham so she's SUSPICIOUS it's more complicated than that. For example, maybe it'll be allowed the second time but not the third, or maybe the second time it gets evaluated by a different rule than it did the first time, but the different rule might not always return 'yes'? Or maybe once the numbers have been used they are allowed in any order?

-1, 0, 1: YES

6, 4, 2: NO

2, 4, 6 for the third time?: YES

6, 4, 2: YES

2, 4, 6 for the fourth time?: YES

2, 2, 2: YES

-2, 45, 1: NO
 
-2, 45, 1: YES

1e16, 1000, 4: NO

1e16, 1000, 4: YES

...on the other hand sometimes the rule is in fact simple and you shouldn't overthink it. 

She submits her guess. 

She's done first. What kind of bizarre nonsense is everyone else getting up to?

Keltham had assigned YES->NO to half the tier-1s and tier-2s, NO->YES to the other half, making sure to put Asmodia and Carissa on separate sides.

He's surprised to find that NO->YES looks like it was slightly easier, rather than slightly harder as he thought it would be.  He would've given NO->YES to the new candidates if he'd known.  Oh well.

Alexandre was the last one, right?  After a quick final round to make sure he didn't forget anyone, Keltham announces that they're done.

"And now," says Meritxell, "all of the slowest performers will be dropped into boiling acid as an example for the others, exactly as it's done in dath ilan -"

"She's kidding," says Gregoria.

"They knew that, spoilsport."

"They might not have known that!"

"I was kidding," says Meritxell. "Only the absolute worst performer is dropped into acid and it is not necessarily boiling, depends whether it had time to get to a boil while you were being slow-"

Since Korva was bizarrely un-slow, she doesn't worry.

She does wonder whether anyone is going to talk to the old guard about whether alter-Cheliax could plausibly have people who might not have known whether several of them were going to be immediately dropped into boiling acid due to slow performance on the very first exercise, or whether that sort of thing is only for newcomers.

There's probably no way to find that out.

"Understood," says Alexandre drily. "Is it permitted to cast Resist Acid first, or does does dath ilan forbid protecting oneself as opposed to the spirit of maniacal-experimentation?"

(He knows perfectly well that countries other than Cheliax are too weak to drop their students into boiling acid, and dath ilan is far more good - that is to say weak - than Taldor. But he may well be whipped once he's finished, anyway, or at least beaten, Taldor does still exist so it can't be that weak.)

Was that appropriately witty and spontaneous or did Alexandre just make some terrible error? She thinks that one student speaking up like that is probably more likely to happen in alter Cheliax, but her trust in her own instincts is at an all time low right now.

At least it's not the type of call she's going to have to make, being 'shy and hesitant.' She hates that she feels relieved.

"I'm not even sure being faster at the exercise is being better at the exercise," the Chosen of Asmodeus, who was the fastest, says in the same light-banter tone. "One way to be fast is to be careless, which is the only real way to fall into vats of boiling acid in dath ilan."

"But today we are charged with doing maniacal-experimentation," says Meritxell. 

"I would of course never dream of working with volatile high-temperature acid without casting Resist Energy (Acid).  It was in no way the case that our Nethysian Safety Officer had to browbeat me into waiting a day to pray for those spells first."

"And today, indeed, we're not just learning maniacal-experimentation, but learning the Law of maniacal-experimentation.  Of this the Law is relatively simple: when experiments are cheap, do lots of them, when they're expensive, spend more on figuring out which experiments to do.  Even then, though, when you're doing lots of similar experiments and they're all failing, what you probably need is to back off and think.  Even even then you might try running some weird or random experiments anyways, while you're thinking."

"Incidentally, a bit of maniacal-experimentation I did myself on the side:  I assigned half the current researchers to start with 'Yes if not tried before' and half the researchers to 'No if not tried before'.  I was expecting 'Yes if not tried before' to be the easier one for non-dath-ilani, but it looks like it was actually the harder direction.  Sorry about that.  Had I run a pilot experiment first, I'd have known to run the real experiment with 'No if not tried before' for all the newcomers, instead of, as I did, it all being 'Yes if not tried before'.  Such is the price of maniacal experimentation where you don't run smaller pilot tests before you run the big ones."

"Carissa was in the 'No' group, but still a huge 'outlier' in speed even so.  Is there some kind of Worldwound experience that let you get it after your fifth input, or...?"

" - after I tried a couple things I figured I should check whether a given input always produced the same output or not. I cannot think of a specific occasion at the Worldwound where I had to do this but it does seem like - sort of a thing to do fairly early in figuring out something mysterious."

"Yeah, that's basically around how long it takes dath ilani kids to get it.  We play a lot of games with 'gotchas' of all kinds.  Even on the non-game side of things, by the time a kid gets introduced to this game, they've probably heard the proverb that when you're trying out new maniacal-experiments in a new domain with new equipment, among the first things you try is to measure the same thing twice, or two things that you expect to be very similar, so you can see around how much error there is in your measuring instruments."

"I wasn't expecting this problem to be as difficult as it was for non-dath-ilani... er, other than Carissa.  I was mostly trying to set you up for the trap where you see 2-4-6 and try 3-6-9 or 3-5-7, and are sure you've found the answer then.  The lesson about that is to test the negative spaces as well as the positive spaces, the places where your model says a procedure should fail and not just where you expect to succeed.  To pin down a boundary you need to check what should be outside, not just what's inside."

"I wasn't expecting the step after that to be as hard as it was.  But it looks like you all ended up learning a valuable lesson anyways, and the 'explicitization' of that lesson is this:"

"Very often in Science, especially when you're working in a confused 'pre-paradigmatic' field, 98% of the work is in coming up with the right hypothesis to test.  That's often more important than the elaborate Law of Probability about how to interpret results that are less than totally clear.  We study that part because it has clearer Law to study and it helps reshape our thoughts, not because it's the most important or difficult part of the problem."

"And of that work of coming up with the right hypothesis to test, again, often the most difficult part is seeing the rule you were taking completely for granted - not a rule you explicitly believed, just a way you behaved automatically without being able to see that and so question it.  As soon as you see the implicit rule, you can imagine it being false, but only once you see it."

"The difficult thing, in most pre-paradigmatic and confused problems at the beginning of some Science, is not coming up with the right complicated long sentence in a language you already know.  It's breaking out of the language in which every hypothesis you can write is false."

That sounds extremely wise; does Keltham intend to explain how to do it?

...she kind of did explicitly believe the rule that was getting in her way, though. Young children probably would be better at this problem, which isn't that surprising because it probably is, in fact, an exercise for children. It's not that they're fundamentally sharper, they've just taken fewer math classes, and don't know that in math classes you mostly deal with the sorts of problems that always output the same answer for the same inputs.

Note to self: next time, before you panic, take a step back and pretend to be a five year old.

There was an expectation buried within Willa, so deep that she couldn't notice it, that real world things might look back at you when you looked at them, but purely math things never did. The thrill of realizing this is sorta wrong is the first thing to really break through her depressive spiral since the class started.

"Breaking out of your language and finding new structures in which it is possible to frame the correct hypothesis, is legendarily the most difficult part of Science to teach; any part of it that's been reduced to a formula is no longer the difficult part of the problem."

"In the case of things like manufacturing acid and improving spellsilver refinement, you'll be at a great advantage because I can tell you much of the correct language to think about those problems... until we start trying to improve those processes using dath ilani knowledge plus magic, as was never in dath ilan, and then we're back to 'pre-paradigmatic' experimentation again."

"Since 'jumping-out-of-the-system' is such a vital part of the Science problem and so difficult to reduce to formula - there is Law of it, but it doesn't help much unless you're making some really basic mistakes - it is mainly taught to dath ilani children by experience."

"For example, dath ilani children are told that the reward of passing a competence test will be theirs, if they can only guess correctly what it is that holds up the Sun in the sky, and prevents it from falling down, and guides its motion, as it circles our planet every day.  Is there some shell that holds it in place?  Invisible wires?  Is the Sun just made of very light material so that it doesn't fall?"

"Of this it is forbidden to speak to children, until they guess it for themselves.  Only then, as their reward, are they shown where they are within the universe - where they have always lived, all their time."

"I assume everybody here was already spoiled on that one because Golarian.  But just in case, raise your hand if you already know."

Chelish wizard students do in fact know that the Sun is very far away and Golarion circles it. 

"In retrospect I should have checked that before showing the current researchers where dath ilan was within its universe.  But looks like no harm done."

"So, yes, until the kid says that they suspect that dath ilan is spinning in place and that just makes it look like the Sun is circling it, while the Sun is staying in place, they're not shown any model of dath ilan's solar system."

"And then, once they see it, there's a new question: what motive force keeps dath ilan circling the Sun?  What keeps the other planets circling the Sun?  What keeps the Moon circling dath ilan?  Why don't they wobble out of place, slow to a halt, or keep going in a straight line instead?  Everyone know that one already?  Raise your hand if you do."

They are taught, too, that it's gravity, the force that pulls the heavenly bodies in their places. (They are not specifically taught why gravity causes that to happen instead of causing some other thing.)

Well that seems like an important dath ilan fact, the not telling anyone about the solar system before they've earned it. How many kids ever figure that out? Approximately intelligence sixteen on average, so, what... forty? Fifty percent? She's still gotta be crap at probabilities for things where you don't have any examples to work from, and she's not sure that the fact that Golarion is spinning even was something that humans ever figured out on their own, so it's hard to say how long they could go without figuring it out, but, hm, she doesn't feel like she could possibly have more than a fifty percent chance of figuring that out alone. Probably way less than that. Maybe if someone were standing by to tell her right away which of her guesses were wrong, like in the exercise just now, so that she could take hundreds and hundreds of wild guesses until she happened upon the right one by being ever more imaginative, but a lot of people still wouldn't, right...?

There could be lots of stuff like that, that people aren't privy to until they've figured out some crucial element for themselves. That fits into what the Chosen of Asmodeus was saying, earlier, about how lies are necessary for most people, but how it's possible for some people to actually be told the truth without the truth making them even more wrong. Maybe you first have to demonstrate that you're the sort of person who can figure out some pieces of the truth on your own, before you can be trusted with more adjacent pieces.

(And she might feel just a tiny bit of pride, about that, not so much for herself, but for the five-year-old who had the impulse to chase the shadows of true things through the pages of library books that were incompetently censored. She wouldn't have to feel so concerned about having that impulse, if the way of knowing true things without breaking was to earn them.)


...except, wait, if you prove it in a context where you can guess as many times as you want, you haven't proved that you'll consistently land on the right answer with the right information at all, you've only proven something about the scope of your imagination, which might be useful for finding right answers, but also seems like it would lead to lots of wrong ones.

Hm.

"You know, I am confused about why anyone would spoil the puzzle for you by telling you that it had something to do with gravity, and then not teach you any of the math you would need to do anything with that knowledge, or really actually know it at all."  Namely calculus.  "I'll have to think if there's anything I want to do with that.  Possibly you weren't spoiled on at least some parts of that puzzle."

"We've now seen what's implicitly the hardest part of the problem that's most difficult to train: picking the right language to think in.  Dath ilani kids get exposed to lots of problems that require 'jumping-out-of-the-system', finding a solution that violates what you thought were the rules, problems that are unsolvable until you look at them from the right angle of sideways, etcetera etcetera.  I'll see what I can do about recollecting some of those and throwing them at you."

"The warning sign that you need to 'jump-out-of-the-system' is the feeling that you've just had.  Well, that everyone except Carissa had.  Frustration, flailing around in the dark, trying desperate wild ideas and getting unhelpful results one after another.  When you feel like that, you're probably thinking in the wrong language, or missing something fundamental, or trying to do something that is in fact impossible.  Or impossible using the tools you have."

"In the absence of that warning sign, if it's plausible you're thinking in the right language at all, the next step is trying ideas from that language, looking for experiments that give interesting results, not just experiments that prove a particular theory, and then interpreting what you see..."

"But first, I conjecture that - although you all continue to look very interested and cheerful, even having finished the task requiring you to control your expressions - you are in fact stressed out by how I was asking you to do something you were never taught how to do."

"All 'pre-paradigmatic' Science is like that, let's be clear.  'Pre-paradigmatic' Science is what you do when you don't know what to do next.  Like the distinction between 'calculation', when you know the next steps, and 'mathematical-thinking', when you need to figure out the next steps.  You are here to invent the straightforward solutions that other people will someday learn, and that is not, itself, a straightforward task."

"But while you are learning to do that, if you ever want, for example, any recovery breaks after I've asked you to do something that was in fact very stressful, you need to actually show any emotion.  Or, failing that, say something out loud about it."

"When I first met the current researchers, I was trying out dath ilani teaching methods that aim to produce confusion, so that people can learn what to do when they are confused.  Instead of looking confused, my students continued to look fixedly cheerful and enthusiastic, as is a Chelish person's 'dignity'."

"So I tried harder to confuse them."

"Bit of a 'communication-with-aliens' problem, there."

"Ione, am I wrong about this?  I was expecting you tell me they were feeling stressed about -"

"They're a bit stressed, but they also think of themselves as tough Asmodeans who wouldn't be happy about my speaking up about it.  And they are in fact tough Asmodeans who'll end up fine even if you stress them out for a few hours more.  We were scheduled to go to the Worldwound, some of the people here are back from it."

"Cool.  I can respect that."

"Regardless of how tough you are or how much I respect that, take a five-minute break anyways, to recover from that and talk among yourselves.  The tier-1 researchers will clear out in the meanwhile, so you're not around anybody too high above you in the org chart.  Ione, Carissa, Meritxell, Asmodia, with me."

Alexandre has been carefully modulating his emotional reactions and letting five to ten percent of them through! He's kind of surprised that Keltham hasn't noticed how obviously he's giving off signs! He stops once Keltham leaves, of course; it's easier for him to think, if he doesn't need to bother controlling his emotions and can just conceal them and focus on what matters.

How could he have solved the puzzle better? How could he have not come in last? (other than sabotaging the other researchers which would get him punished and alterAlexandre would never do and so he is at no point seriously considering.) He can name the unspoken assumption he made now, that math never reacted to you, but the math was supposed to be a simulation of nature, which can react to you. (As in Keltham experimenting with acid and needing protection spells.) If he had paid more attention to the instructions, he might have noticed the confusion. That he didn't was an error characteristic of underestimating Keltham. An error he had been warned against and made anyway.

(He is still kind of expecting to be whipped for that; he can understand smart mistakes not getting you whipped, but stupid ones?)

Either way... so the warning sign is that he can't think of anything to try. In practice he could think of things to try, prime numbers, size caps, base-12, individual orders of magnitude, but all of them were unlikely. They were inelegant. They depended on Keltham having arbitrarily designated some range of numbers as 'acceptable', instead of having a very simple rule that was just outside what he could imagine.

The key thing to search for is the mental state. His frustration. The sense that he was beating his head against a wall.

Well. He can look for that.

(... Also, more worrisome: Every dath ilani child can solve the mystery of the planet's motion? More evidence that he's in over his head, as if he doesn't have enough. He is starting to understand his superiors' frustration, and to be impressed at their accomplishments. He will need to work harder.)

The kid really does think infuriatingly highly of himself.

Not that he's wrong to; it is clear why he is worth as much as he is to Cheliax, and probably any random peasant who suddenly found himself treasured by the Crown and fought over by gods themselves would comport themselves similarly. She worries, though, that it's contagious; part of what Keltham is trying to impart is the habits of dath ilan, the attitudes of dath ilan, and Keltham is a teenager who thinks too highly of himself, and so he'll teach them all to become teenagers who think too highly of themselves. 

You should never get too attached to a single frame through which to view a person, but as a first pass it seems to explain Keltham and Asmodia both; the Chosen of Asmodeus, she does not dare assess in the same terms, not exactly. 

The classroom is frightfully silent, probably because everyone is waiting for her to say something. Well, she'll let them wait a bit longer; she's not here to make them feel like there's someone competent around, even though obviously she is having that effect.

At Ostenso, Pilar was often the first person to talk during a Silence.  A little annoying, maybe, but she understood why it had to be her.  "If Keltham asks later, Willa, Alexandre, and Tonia got cookies from me.  I get why nobody in realCheliax wants one right now, but those reasons don't apply in alterCheliax."

Korva doesn't say anything to anyone, because she doesn't have anything to say. She doesn't think about the problem, either, or how she could have solved it faster - she hasn't made it through all of the transcripts that were given to her, yet, but she's definitely noticed the pattern that Keltham will constantly say things that appear to be hopelessly convoluted and complex, before more information reveals that he's actually saying something very simple, and that she wouldn't have gotten if she had been busy trying to figure out what it could possibly be with only the information she had so far. So she'll wait, and see if there are any later clues that indicate that she hasn't fully understood the lesson that's been taught to her.

Korva privately supposes that there are actually lots of different reasons why someone might not want a cookie.

"Also most of the reasons the atmosphere in alter Cheliax would be less funereal ought to apply here and you could all stand to relax," Gregoria says. "That went fine. There'll be homework, and how you do on it will matter much more for Keltham's impression of your intelligence than this. So you're not Sevar; you already knew that."

"Is she - always much more skilled in everything," Sibilla asks.

"If so, she doesn't always show it? Asmodia usually finishes the purer math stuff ahead of her. You're definitely allowed to be faster if you can. In alter Cheliax there's no Chosen of Asmodeus."

"Is Keltham under the impression she has the same standing as his other students? He indicated she was responsible for the nametags, if they didn't happen - has he not noticed that will obviously make the nametags happen?"

" - you know, I don't know?" says Gregoria. "I think he says 'this is Carissa's problem if no one else solves it' because he doesn't think highly of the project staff and he trusts her? But possibly the project logistics staff in fact work a lot harder when he says that because they don't get as many lectures as we do about acting like they're in alter Cheliax."

"If they have not been instructed better they are definitely doing the idiotic thing."

"Is she not officially 'his deputy'?" Alexandre asks, having memorized the org chart last night so he could know he officially had to defer to. "And the 'leading shareholder' not from dath ilan?"

"Correct."  She read the corporate contracts a few times because alterPeranza likes Civilization things.

"That fails to solve your problem, which is that anything Keltham deputizes will get done as quickly as if his deputy were really in charge of this whole operation. In alter Cheliax I cannot imagine logistics would be tripping over themselves to help alter-Sevar."

"Alter-Sevar is still sleeping with the Queen."

"Can everyone extrapolate my response or need I say it, possibly in more filtered company."

- Gregoria has honestly half lost track of what they're arguing about. 

Willa is unhappy that alter Willa needed a cookie.

Outside of that, five minutes isn't a lot of time for Keltham to get safely far away and back, so she's going to stay firmly in her shell, just in case. Even if it is, she isn't really feeling gregarious at the moment. She'll let the conversation about Sevar mostly flow over her.

What she wants to do is use mage hand to play with her quill; it's one of her favorite ways to destress (and won't it be pretty to watch once she gets that Arcane Sight.) But doing that would probably draw attention that she doesn't want to deal with. It might not even be adequately shy. She'll have to stew without assistance instead.

Willa wants to be special, and she's made negative progress at that since getting on this project. She isn't especially good at the work (yet), she couldn't make clever statements even if she could think of any, and most of her superiors already dislike her. She feels frustrated, like she doesn't have the right tools to solve the problem. Alter Keltham who she could whine to about this problem would tell her to step outside of the solution space.

What's special about Willa? If she gets hired, she's going to be selling her soul; that isn't special, especially not in Cheliax, but also selling an option on her sold soul is. Is it useful somehow? She can't think of how, especially since all the other new hires will be in the same boat, but it's a persistent idea all the same. Selling an option on a soul feels like it was definitely someone stepping outside of a solution space, but as far as she can tell it's not to her benefit. Hell is going to own her soul; in all respects it should probably act as normal until and unless the option is executed?

What other especially unusual stuff has happened? People keep ominously mentioning the tropes, so they're probably something special, but without knowing anything else about what they are they aren't special in any useful way. Even the 2-4-6 'math looking back at you' angle doesn't help if you have no idea what you're trying to look at.

Alexandre thinks about telling Lady Eulàlia Avaricia de Seguer that, actually, no, in an alter-Cheliax that was at all competent, logistics would be tripping over each other to help Keltham's deputy, because literally every instant of Keltham's time is worth more than the lives of several dozen ordinary people. It's an enjoyable fantasy which would be highly unwise for his future life expectancy to try to execute, so he isn't going to do it. He's just going to think about it.

...and everyone's quiet again.  She'll give it one more try.  "Korva, you finished faster than anyone who wasn't tier-1.  Any advice that carries over to the rest of us?  This isn't a normal competition, the whole Project succeeds or fails as one and you end up a Duchess if it succeeds."

Korva has literally zero desire to be a duchess, which does not at all mean that she doesn't separately want the project to succeed.

Also she kind of hates talking to people, but that doesn't mean she can't answer direct questions.

"Honestly I think I got lucky? But I thought afterwards that I would have gotten it faster if I'd been a small child, because I wouldn't have taken as many math classes and wouldn't have learned the implicit rule that math classes almost always deal with problems where the same input always gets the same output. And since a lot of the people in this room were specifically picked for their ability in math classes, they might be even more vulnerable to that than a normal person. So I was thinking that the next time I ran into a problem like this, if I got stuck, it might be worth pretending to be five, and seeing if it helps anything."

The problem with saying things is that most things make you look weak, foolish, or low-status. So, useful information, Korva! We're all better off for you providing it! Alexandre is keeping his mouth shut.

Willa's model of her five year old self has a lot of really obviously terrible ideas about how to try to be special here.

And Keltham is back!  He doesn't ask what transpired during his absence, so nobody has to scramble to make anything up.  He does ask if anyone's got any brief questions before the next lecture section.

Message on relay from Asmodia to Lady Avaricia:  Security forwarded your observation to us.  Mostly, Sevar and Maillol and Security and myself are already on task for deciding which Keltham-requests get handled at what speed.  Nametags will arrive shortly before the hour is up.  But your point is a valid one and I'll ensure that staff here understand what Sevar's alterCheliax priority should look like absent other instructions, namely less than Keltham's and higher than any other project member's.

Lady Avaricia's immediate thought in response is that that this is satisfactory behavior from Asmodia, though she does not expect that to be relayed to Asmodia in light of how Asmodia is technically her superior here. 

She has no brief questions for Keltham.

...no questions, huh.  All right then, onwards.

When Keltham was inquiring into spellsilver refinement, he was told that among the things they'd tried were:

- Using the ash of bones from different animals, and some varieties of people, in case those had more potent alchemical properties than the ash of ordinary random bones.
- Prayer.
- Using fire instead of acid.
- Having a step in the process performed by 'godrelated people' - possibly clerics but why didn't they just say that, well, anyways.

Alas and alackaday, none of this worked to improve spellsilver refinement.

The students here may now recognize this as a variant of their own frantic flailing on the 2-4-6 game.  To be clear, it's much more respectable and dignified to frantically flail around with random experiments in your current broken 'specification-language' of experiments, than to not try anything.  They would have been ill-served, in the 2-4-6 game, by pulling back after their tenth failed attempt to do nothing but think; the Keltham-Environment was still walking around, and there's no point in not running tests when the Environment is right there and cheap to query.  Still, if you imagine 'YES' as meaning 'YES, that didn't improve spellsilver refinement', everything the refiners tried got the same old frustrating YES.

When you haven't yet found the Answer Outside Your Previous Language, the brilliant hypothesis to test - what can you possibly do besides frantic flailing at a process that just keeps on returning the same answer?

Some of the candidates here did figure out something to do besides just frantic flailing, as they zoomed in - or it looked to Keltham like they were following that track, anyways.

"Not sure if Keltham explained this yet, but his rule is that students recognized as more advanced speak later.  So none of the current researchers will try answering that until some of the new candidates have spoken."

"Keltham will also be more impressed by wrong answers than no answers, especially if the silence has gone on for more than a few moments."

"Affirmed, thanks Ione."

"Check whether you're asking the right question," says Korva, because she needs to speak up for the first time at some point. "So - for the spellsilver question, take a step back and double-check whether there are any other ways of obtaining more spellsilver besides improving the existing refinement process."

"Not the answer I had in mind, but a fair one!  I did in fact check whether it was possible to detect giant masses of pure spellsilver from a very long distance away, in which case I'd have some ideas about where to look for those elsewhere inside a stellar system, but no such detection method was known."

"Alexandre," (said after reading the nametag, of course) "by the time you were finishing up, it looked to me like you were no longer just looking at my 'YES' answers, you were thinking about something else too.  Am I correct?"

He does not think Keltham means his flailing with prime numbers or powers or base-12, which means Keltham already knows -

"You are," says Alexandre. "I was paying attention to your response time. You responded faster to some queries, slower to others. Since all questions received the answer 'yes', I gained almost no information from a 'yes', so instead I focused my investigation on what triggered an unusual response from you, in the hopes that would lead me to the Law."

"Well stated.  The reason being, of course, that as there were more and more inputs on your paper, I had to scan through more and more of them to check you weren't repeating anything, especially if the input was just a few small digits with a particular pattern of positive and negative values.  I could, of course, lengthen all of my response times equally, to eliminate the information from this 'timing side-channel attack'.  I elected not to, because it was illustrating an important point."

"From the standpoint of the Law of Probability, any time you see evidence that's almost entirely what you expect, it's not producing much of a shift between hypotheses.  You were already expecting the 'YES' answers, so that part wasn't informative.  You might not have had the correct hypothesis about what was producing the timing differences, to see that it was being validated; but at least you weren't already sure of what the timing would be like on each input.  By poking around there, you were gaining information faster than you could gain it from my, apparently, just always answering 'YES' to everything."

"Willa, you're one of the others who noticed the 'timing side-channel', or so it looked to me.  How would you apply a similar kind of reasoning to spellsilver mining?"

Wow this is terrifying. She obviously has to think fast here.

"I would shy away from trying new tests similar to tests that got a result that was just as good, or almost as good, as the usual process. I'd focus on ideas where the first test went really badly and screwed everything up somehow, and try to do something slightly different than that. Because if it managed to make a lot of difference, even if it was bad, doing something similar might make a lot of difference that's good."

"You'd need to be pretty wealthy to do tests like that because you'd waste a lot of ingredients and potential spellsilver that way, though, and maybe never get anything to show for it."

"Somewhat valid?  It's usually not true that, if you can make something a lot worse, you can reverse the same manipulation to make it a lot better, especially if you're dealing with processes that somebody already refined to near an optimum.  Lots of things that aren't at all like the current refining process, won't work at all to refine spellsilver; if you find that result, it's not surprising."

"The version of your idea that I'd suggest is to look for things you can do that make spellsilver refinement slightly worse, rather than wrecking it completely.  Then, measuring quantitatively how much more of those things you can do, to make the output how much worse.  Then, trying to see if you could predict those quantities - maybe not in the sense that you would have a brilliant theory about it, but in the sense of trying some new intervention that ought to make things slightly worse, and trying to call in advance how exactly that would go."

"Similarly, if you'd been mounting a more serious attack on the Keltham-Environment, your obvious next step - from a dath ilani standpoint - would be measuring my response times down to the second, or even a fraction of a second, writing those down, and trying to predict them precisely."

"This already is a way of thinking that seems not quite to be known yet in Golarion.  The spellsilver miners and refiners to whom I spoke, had stories of what didn't work, but not measurements of exactly how much it made things worse, nor theories to predict exactly how much damage would be done by which failed interventions."

Willa feels chastened; she was so eager to 'step outside the solution space' that she went too far outside it. And looked stupid again.

But it occurs to her now that even if you were trying relatively risky experiments, you might not have to actually try them all: you could just decide on what experiments you might do, and then use Auguries or even more powerful future predicting divinations to see if they would lead anywhere or not. It'd be a little bit error prone, but it'd also be much cheaper and faster than going without.

This isn't to say you wouldn't still do experiments and take data, but the experiments you ended up doing would be selected in advance by a sort of filter. You'd effectively be putting more educational sequences into your 2-4-6 game, on average.

This is an idea born of magical knowledge though, and so it might be more than a little dangerous. Especially because getting Keltham into the habit of preparing and using too many divinations of that sort could end up going pretty badly. But then again, it's the kind of thing Keltham will probably think of doing himself before too long, and it might make the conspiracy universe more probable if all these other people more used to having magic available didn't suggest it first.

She thinks loudly at security to ask Asmodia if she thinks mentioning this Augury-pre-experimenting line of thought is worth the risk.

Sevar's call.  Considerations Asmodia can see include 'if he relies more on Auguries we can spoof him', 'it adds a random factor that makes it difficult to control him', 'we screw up the probabilities on Augury failures', 'if this speeds things up we want his work sped up', 'if this is a good technique we want it developed here and not in Osirion later'.

She'd be firmly against, if not for the ability to spoof the augury; given that, it seems worth proposing.

Willa manages to be surprised that she gets to ask the question herself; she'd kind of been assuming her questions would go to someone else if they somehow managed to be good. In hindsight, that would probably create bad colors on The Wall, which is something she should've thought of to begin with.

"What if we used auguries to help select our experiments? We would think of two plausible experiments to do every time we would have done one, and then cast augury to pick which."

She's getting pretty excited now and her words start speeding up. "It wouldn't always select the more useful one, but it would more often than not! So the data we'd get would probably end up being more useful! We could make progress faster, and we could even try slightly riskier, costlier experiments, since the worst failures would usually be filtered out by the auguries!"

"Auguries are second-circle spells and trade off directly against Resist Energy (Acid) spells; their time horizon is roughly half an hour; and they're not reliable enough that we could try anything we had a strong prior expectation would be dangerous, even if otherwise cleared by Augury.  It's not a bad idea, but relatively narrowly applicable - we apply it only when the experiment is expensive enough that we don't just do it regardless, only when the time horizon is less than half an hour, only when the danger level is such that the Augury's 'likelihood-ratio' of around 4 actually shifts our decision."

"We'd also have to experiment earlier to see if Augury could learn to define 'got a surprising or enlightening result' rather than 'improved the manufacturing process' as being the 'good' outcome.  Which is my way of changing the subject back to the next point I was going to make:  Civilization teaches a soft conceptual separation between understanding a phenomenon, and improving it.  We want to develop magical Science!, and it's tempting to jump right to casting Auguries as a way of improving it.  But first we'd want to understand how Auguries interact with the work of Science."

"Can Augury define 'got a surprising or enlightening result' rather than 'improved the manufacturing process' as being the 'good' outcome, if that's what we want?  Is an Augury smart enough to know that a very surprising explosion can be worth the cost of equipment destroyed, for what the explosion tells us?  Do we have to figure that out within half an hour for the Augury to know it was worth it?"

"The first step would be to do promising experiments, cast Auguries before doing them, go on to do the experiment regardless of what the Augury said, and figure out exactly what Auguries were telling us."

"The question that Science begins from is 'How do we understand spellsilver refining?', and it is then this understanding that we use to ask 'How do we improve spellsilver refining?'"

"The heartbeat of cognition, as it would be said out of dath ilan, is that sensory information floods into you on the 'diastolic-beat', from this you build a map or model of reality, then you search for strategies that the current model predicts may succeed, then actions surge out of you on the 'systolic-beat', which actions affect the environment and the 'sense-data' you get on the next heartbeat.  Where of course it's not a 'pointwise-perfect' analogy because in your heartbeat the expansion that draws blood in, and the contraction that pushes blood out, are distinct phases.  In cognition, sensory information is always coming in and we're always acting.  But, you get the idea.  Well, hopefully you get the idea.  I'll actually just spell out the idea.  The idea is that we can be overeager about jumping ahead to trying to improve things, not so much before we understand them, but without having properly realized that understanding is a distinct task that we can focus on and solve in its own right."

"Consider again the spellsilver refiners; if they'd been saying to themselves 'How can I figure out exactly what's happening during the spellsilver-refining process?', they might have performed different experiments and taken more precise notes.  Contrasted to asking 'Does this work to improve refining?' and then, when the answer is 'No', they treat that as a failure."

"The same distinction reproduces itself on the meta-level; you should consider yourself as first asking, not 'How can I make Science work better?' but rather 'What are the rules governing which ways of pursuing Science would yield which sorts of things happening?'"

"If you're just running ahead trying to improve things, you might think that 'map first, then improve' sounded like great advice.  Once you enter the mapping mindset, you'll realize that the key claim is more like, 'people who follow "map first, then improve" ultimately invent improvements faster than people who don't follow that advice'.  That makes it plainer that this advice is really a theory, and a theory you could poke at and test to understand the laws governing people using Science; then, with this understanding, try to improve your pursuit of Science."

"This process of Science reflecting and improving on itself is not, to be clear, something you can continue without limit until it turns you into a god.  There's 'diminishing-returns' and the process 'asymptotes', at least as humans do it in dath ilan.  Once you're doing most basic things right, doing things slightly more perfectly than that is often not worth the extra time it takes to think.  But that process of reflection is how you get to the point of doing the basics correctly; and that's not a process Golarion has undergone yet."

Well, the sort of tedious people who make spellsilver for a living haven't undergone it yet; Lady Avaricia thinks she'd do fine at it, if she were going to work for a living.

"...what's a diastolic beat?"

"...your heartbeat has two distinct components, a 'diastolic' beat where your heart opens valves to your 'veins' that carry blood inwards towards the heart, and expands to draw in blood, then a 'systolic' beat where your heart closes those valves and opens valves to the 'arteries' that carry blood outward, and contracts to force the blood it drew in outwards."

"Or in more detail, it's really more of a four-stage process where blood gets pulled inward from your body, pumped outward to your lungs to get Element-8 restored from the air you breathed in and exchange 'two-8s-one-6' to your lungs for you to exhale, pulled back into your heart, and pumped out to the rest of the body, except that the two pulling-in stages from body and lungs, and the two pumping-out stages to body and lungs, happen simultaneously even as blood flows through four different pipelines... I can draw a diagram if for some reason it's important."

"I think I'd actually rather return to the thing it was a metaphor for: sensory information coming in to our brain-hearts from the environment-body, us thinking, which you could see as a kind of lungish metaphor thing, actions going out of the brain-heart to the environment-body, interacting with the environment, and we then see what happened when the sensory information comes back in.  Only as a continuous process rather than in distinct stages -"

"You know, actually, forget that entire metaphor."

"Just remember that improving things and understanding things are distinct tasks, and you can often make more headway by separating out the 'understanding' part so you can explicitly pursue it as a task in itself."

"Does the distinction between understanding and improving correspond to the distinction between the Law of Probability and the Law of Utility?  It sounds like it should."

"Sensible question, but no, not exactly.  Probability is something like a separable core that lies at the heart of Probable Utility.  The process of updating our beliefs, once we have the evidence, is something that in principle doesn't depend at all on what we want - the way reality is is something defined independently of anything we want.  The scaffolding we construct between propositions and reality, or probabilities and reality, doesn't have a term inside it for 'how much would you value that thing', just, is the coin showing Queen or Text."

"But the process of Science, of experimenting on something to understand it, doesn't belong purely to Probability.  You have to plan experiments to find ones that distinguish between the possible hypotheses under consideration, or even just, are effective at probing to uncover surprises and unexpected patterns that give you a first handle on what's happening.  The Law of Probability just says how to update after you get the evidence.  Planning an experiment that you then act on, implement, is the domain of Probable Utility and can't exist apart from it."

"In fact the influence of the 'utilityfunction' on 'epistemics', the influence of what we ultimately want on how we map reality, is in-theory-but-not-in-practice much more pervasive.  In principle, how we classify things in reality and lump them together - treating all gold pieces as 'gold pieces' instead of as uniquely detailed individual elements of reality - reflects how any two gold pieces are usually equally useful to us in carrying out the same kinds of plans, they are plan-interchangeable.  In practice, even people who want pretty different things, on a human scale, will often find pretty similar categories useful, once they've zoomed into similar levels of overall detail."

"Dath ilani kids get told to not get fascinated with the fact that, in principle, 'bounded-agents' with finite memories and finite thinking speeds, have any considerations about mapping that depend on what they want.  It doesn't mean that you get to draw in whatever you like on your map, because it's what you want.  It doesn't make reality be what you want."

"But when it comes to Science, it really does matter in practice that planning an experiment is about wanting to figure something out and doing something you predict will maybe-probably yield some possibly-useful information.  And this is an idea you just can't express at all without some notion of Probable Utility; you're not just passively updating off information somebody else gave you, you're trying to steer reality through Time to make it give up information that you want."

"Even when you do get information passively, figuring out what to think about it reflects which thoughts you expect will be useful.  So the separable core of Probability inside of Probable Utility is really more of a Law thing about basic definitions, then anything that corresponds to - there being a sort of separable person who only implements a shadow of Probability and doesn't shadow any structure cast from Probable Utility, who's really great at understanding things and unraveling mysteries and answering questions, but never plans anything or tries to improve anything.  Because humans are constantly-ubiquitously-in-the-unseen-background choosing which thought to think next, in order to figure things out; usually wordlessly, but in words too when the problems get especially difficult.  Just the action of turning your head in a direction, to look at something, because you wordlessly anticipate gaining info that has the consequence of helping you answer some other question, is in theoretical terms an action."

"Just to check, is that supposed to be some kind of incredibly deep lesson full of meaning about something else important?  If so, I didn't get it."

"Nah, it's just an answer to your question.  Or at least, if it had some hugely important hidden meaning about how to avoid some dreadful Science!-related catastrophe, I didn't get it either, when it was emphasized to me as a kid."

"Anyways, I think we're drifting off-topic a bit."

"Returning to the main point, I suggested that if you were mounting a serious assault to conquer the Keltham-Environment, and you hadn't figured out yet how to get any result but 'YES', your next steps should include:  Measuring other things about me besides my 'YES' answer; measuring how much time I took to answer each input, as precisely as you could with your time-measuring instruments, and writing down all those results; trying to predict the time for new inputs you constructed."

"I don't know how much Probability you got off Asmodia, but can any of the researcher-candidates say what that advice truly means - why it makes sense - in terms of the Law of Probability?"

"Because it - removes twos; it destroys some of the probabilities, by exposing them to reality; it alters the ratio of the masses on each of the thousand sides of the scale, by evaporating the weight on them unevenly between them. The more we see that some types of questions seem to take longer for you to answer, the less likely it is that the time it takes you to answer a question is unrelated to the contents of the question."

"Every time you expose yourself to the reality of my 'YES' answer, if you weren't absolutely sure already that you'd get a YES answer, some of the probabilities evaporate.  That's already true.  What's different about adding the timing measurement, then?"

"How does the evaporation of probability change with measuring my timing?  How does it change with measuring my timing more precisely?  How does it change with trying to predict the timing for particular inputs instead of trying random inputs and passively observing the timed results?"

After receiving mental approval, Willa says, "I was really surprised by the first long pause after an input of very simple small numbers, then I tried bigger numbers and got a short pause, then I tried small numbers again and got another long pause. This seemed to sharply reduce the probability that whatever was being done to the numbers was a direct computation of some kind, and from there I realized the type of answers I'd been missing really fast."

"The leading theories I had were that long pauses meant difficult questions, long pauses were unrelated to the difficulty of the question, perhaps being randomly decided, and long pauses were a deliberate attempt to mislead me, 'trolling' in your parlance, in that order; I would have put roughly a 6:2:1 ratio on them, when I started, though in fact I did not think that explicitly. If I asked a question that I expected would be difficult and you answered slowly, that was evidence that either I was right it was difficult or you were trolling me, and also that - given that I believed it was a difficult question - that my first theory was right and my second and third theories were wrong.  If, on the other hand, I expected that it would be difficult and you answered quickly, that was evidence either that I was wrong about it being difficult or I was right and you were trolling me, and also that my leading theory was wrong and hence one of the others - or neither of the others - was correct."

"So that's the particular things that happened during your 'timing side-channel' experiments, and what you thought about them in Taldane.  The thing I'm asking for is more like -"

"Well, I called it Probability Sight at one point, by analogy to Arcane Sight, and Asmodia called it that too after she suddenly developed it one day."

"It's an abstract way of seeing things that would generalize to doing more Science later.  I'm trying to figure if there's some way to get it into people without either raising them as dath ilani, or dropping an artifact headband on them for two hours and inducing a permanent need for a cognitive prosthetic."

"Let me try -"

"So, a few days ago, when I was introducing the idea of lost-powers-of-2 as the 'truth-functional' relationship between a probability-claim, and reality, Carissa was briefly worried about how, if you could only ever lose 2s by playing that game, would that mean, people would be incentivized to avoid playing it?  And I was like, nothing bad directly happens to you when you lose a 2, it's just a relative measure of how well you're predicting, given that you're trying to predict at all."

"When you start trying to predict the timing, you lose more 2s, or 'bits' in Baseline.  The joint probability you assign to everything you see, goes down, because you see more.  But in the course of losing more 2s, you further narrow down what's probably true, and concentrate the remaining probability that hasn't been eliminated."

"This is not necessarily something that happens every time you manage to predict more and lose more 'bits'.  If that was always a productive activity, I could go on spinning a coin and seeing whether it came up showing the Queen's face, or the text on the reverse side, and lose one 'bit' every time.  The thing about the 'timing side-channel' is that some different hypotheses about what could be going on, lose different amounts of 2s; that's what makes it an informative experiment."

"That's why adding this new 'variable' to measure and predict, which makes you lose more 2s, is actually a great move, because the 2s are differentially coming from different hypotheses.  Figuring out which hypothesis is true, is what we care about, not how many 2s we lose along the way inside predictions we didn't stake lots of value on getting right."

"Why does it help to measure the times more precisely?"

"Because it's more specific?" 

"Can you be more meta-specific about exactly why it helps to be more specific?  In the language of Probability?"

"Because the more precise your claim is, the fewer twos you lose if you get it right and the more twos you lose if you get it wrong?

"I was asking about why it helps to have a more precise measurement, not to make more precise claims.  But maybe you're right and all we really need is the precise claim.  So we should measure my response times to within the nearest 6 seconds, but predict that my next response will take exactly 4.59823987 seconds.  This way we get all the benefits of a precise claim, but without the risk of losing lots of 2s by actually measuring.  Would you agree that this is the pathway of wisdom?  If not, why not?"

"More precise answers rule out more things? Like, if we're looking for a person, and we can measure where he is but only to within five hundred miles of precision, our measurement moves us towards all the theories that said he'd be in Cheliax, but can't tell the difference between theories that said he'd be in Egorian and theories that said he'd be in Ostenso."

"You're not actually gaining more information from the precise claim if you can't measure whether it was truer than a less-precise claim."

"This time it would've helped because if you kept taking data it would've been clear the pauses were slowly growing longer, on average, as more and more sequences were tried. After long enough, any explanation that didn't have a reason for the pauses to grow longer wouldn't predict that, and it would lose 2s."

"Because 'short' or 'long' is yes-or-no, but fractions-of-a-second are continuous; you can only get one bit out of a yes-or-no question, but many more out of a continuous one. The more data you have, the greater the updates you can make based on it."

"Having lots of precision can't possibly help - I mean, the difference between 4.59823987 seconds and 4.59823986 seconds - that difference isn't going to be related to anything -"

"I mean, it totally can help by the time you're Civilization, but you're not gonna get there for a while.  That's the kind of precise difference you'd run into if you were measuring the difference in exactly how fast time flows at the bottom of a mountain versus the top of a mountain."

"And, okay, let me finish responding to some of the previous things."

"You can't literally measure a continuous quantity and get a continuous answer, in fact, because that would correspond to infinite precision and an infinite number of 'bits' in the measurement.  It's also not true that you can only lose at most 1 'bit' on predicting something yes-or-no, for instance, you could predict YES with probability 63/64 and NO at 1/64 and see NO and lose 6 'bits'."

"When it comes to noticing just that the pauses are getting longer and longer, just measuring to the nearest round works for that; you don't need to measure to the nearest second.  Being able to notice the pauses are getting longer, is an argument for measuring the timing at all, because it causes some hypotheses to differentially lose 2s.  It's not an argument for measuring precisely."

"Among the remaining arguments, we have:  An argument that precise claims help by allowing right claims to lose fewer 2s, and wrong claims to lose more.  An argument that, if you don't measure precisely, you can't verify or falsify precise claims.  An argument about how precise measurements can rule out more theories, because if you can see closer than five hundred miles you can distinguish between claims about Egorian and claims about Ostenso."

"Those arguments are all valid, for having been spoken in Taldane.  But can anyone rephrase it more into the language of the Law of Probability, to fit it all into a common framework?  Tier-2s may now also answer."

"A more precise claim is one that - describes a smaller set of worlds," says Gregoria. "Not all the worlds where the person is in Cheliax, just the ones where they're in Ostenso, or just the ones where they're in this fortress. More precise claims are going to take more 2s to - find - and more observations are going to be evidence, about a more precise claim. Almost nothing I see is evidence about whether Asmodeus is 'great', because what does that even mean, but lots of things I see are evidence about whether Asmodeus expends more resources on the Material than other gods, and still more things I see are evidence - no, that's not precisely it, 'Asmodeus has thirteen tines in His crown' is a precise claim but almost nothing has bearing on it -"

" - okay, I think there is something there, I just need to nail it down. More precise claims point at smaller pieces of reality. It takes more twos to point to smaller pieces of reality. A theory that makes a more precise prediction is much more valuable, if it's right, it hit a smaller target. A theory that makes a more precise prediction is more easily wrong, which is just another way of saying what I just said - it couldn't be more valuable if it was right unless it was more easily wrong. You have to have a much better understanding of reality to make more precise predictions and have them still be correct. 

And your claim is only as precise as - if my measurement can't tell the difference between two claims, then I don't get any of the benefits of having made one rather than the other, a claim is only usefully precise if it's precise about what you're going to see, so, what your measurement is."

"I think possibly he's just looking for us to rephrase in terms of Problem 5 from that list of seven problems he gave us just before the artifact headband got dropped on Asmodia.  Which if I remember right was like - if you divide claims into a thousand parts and predict it down to the thousandth, that's like predicting that exact thing ten times as strongly than if you only say it's in a hundredth, but it could be in any of ten thousandths inside that and you're not saying which.  If you only measure it down to the hundredth, you'd - uh, I'm not sure what happens then, but you obviously can't be proven correct in the claim about thousandths."

"Yes, that's what I just said," Gregoria says.

Yes, that is EXACTLY what Alexandre was trying to say!

"Tier-1s."

"It doesn't take more 2s to point at smaller pieces of reality, it takes more 2s to point at smaller pieces of the probability distribution.  If you assign probability 1/64 to something, it takes you 6 'bits' to point there.  If you're already assigning 98/100 to some particular measurement coming in at exactly 0.891 seconds, you're only going to lose 0.03 2s each time.  Once you can exactly predict the measurements very certainly, they're not helping you narrow down things much further.  It would only automatically take more 'bits' to specify narrower pieces of reality if there was - some kind of - fixed probability distribution, or - this actually feels like it's pointing somewhere important but I don't know where yet."

"Precise claims don't have to talk about a smaller set of worlds, there can still be probability everywhere, it's that most of the probability will be concentrated in a narrower set of worlds."

"But theories only have so much probability to spread over all the possible precise measurements, so when there's more possible measurements, the probabilities on the vast majority of possible measurements have to be thinner.  Measuring things to three decimal places is one way to get lots of possible outcomes you're measuring over.  But it could also be something like - measuring three different things about it down to one-tenth apiece."

"If one theory puts lots of its probability within 0.002 seconds of 0.891 seconds, and another theory says 0.887 plus or minus 0.003 seconds, they've got some overlap, but measuring down to the nearest thousandth is pretty likely to do a good job of prying them apart.  Measuring down to the nearest hundredth instead, would be like adding up all the thousandths closest to that hundredth, to get the theory's predictions about what the measure would say as opposed to what was exactly real.  And then the two theories would give around the same probability to measuring 0.89, if you were only measuring down to the hundredth, and measuring at that precision wouldn't pry them apart much."

"You could maybe pry those two difficult theories apart more, even with only one hundredth precision available, if you constructed your experiment to measure them somewhere different."

"Theory 1 might say '7, 8, 9' takes 0.887s +/- 0.003s and Theory 2 might say '7, 8, 9' is 0.891s +/- 0.002s."

"But then you could also do an experiment at '107, 118, 129'. Maybe Theory 1 says that's 0.567s +/- 0.003s and Theory 2 says it's 0.128s +/- 0.002s. Your hundredth second resolution can detect that difference easily, it'll make one (or both) theories lose a lot of 2s in every measurement."

"You want to plan your experiments so they measure a place where your theories with the highest prior probabilities diverge more, so they're easier to distinguish from each other."

"Good!  I'm not sure how to explain what I was looking for, there, or why I wasn't already satisfied with the earlier answers, but it's something like - the difference between thinking in Taldane, and justifying those thoughts afterwards using the language of Probability, versus starting to see in probability distributions, and putting those thoughts into Taldane."

"With respect to the last point: yep, even if you can't measure precisely, you can still go looking for some other place where the major theories come apart.  The thing is, that often requires more extreme conditions, or weird conditions.  If you can measure time using clocks accurate to one part in ten million billion, sixteen digits in base 10, you can notice a clock ticks very slightly faster after you raise it from waist height to head height.  With less accurate clocks you'd have to find much much more extreme, expensive conditions to start losing ten million billion times more time from the same phenomenon, so you could measure missing seconds.  Even Civilization couldn't do that; the energies are too high."

"Still, that segues well enough into my next question.  Why try to predict the timing of my next response?  Why not just observe it passively and consider what it meant only after the fact?"

"If you were a god you could probably learn just as much from observing it."

"Perhaps.  Why?"

"- it just seems like anything you do only in your head, like making a prediction, a god could just - have part of them that didn't have the answer do that, or more realistically probably just do that after the fact but not be influenced by knowing the answer. But humans are weak and flawed and so would err if we tried that, and predicting beforehand makes us less vulnerable to that error."

"I remember `3. No empirical theory can prove itself except by risking its disproof.'"

"Is part of it that if you don't make the theory before the fact of seeing the evidence, it can't have a real risk of being disproved by that evidence? Because if the evidence was already there you knew how it would interact with the theory beforehand, and so there couldn't be a real risk involved?"

"More of an issue with mortals than with 'ideal-agents' or, I suspect, gods.  Or maybe it's just the Lawful ones, I don't know.  But an 'ideal-agent', even a 'bounded ideal-agent', if It considers the theory after seeing the evidence, won't be at all influenced by having seen the evidence, in what predictions It says the theory makes.  So while It might not even bother to invent some wrong theory, if the evidence contradicts it, it would nonetheless be the case, that if you asked It what the evidence said about that wrong theory, It would know the correct answer there."

"I nonetheless claim that if you hook up to the Keltham-environment a random input generator that picks three random numbers each independently between negative one thousand and positive one thousand, and let a god passively observe that, the god will learn less than a mortal playing the game with Keltham.  Well, maybe not with the 'timing side-channel' information added, but definitely without that timing information."

"Furthermore, I claim that this mysterious superiority of the mortal, over the god, has to do with a process which includes the mortal predicting things in advance of seeing them, nor may it be done without the mortal making those predictions."

"Well, say the god has several theories and isn't sure which is right. If the god doesn't get to choose the numbers then they can't pick numbers that distinguish between their theories. You can figure things out a lot faster if you get to choose what tests you're doing. So being able to choose which tests to do is really valuable, but only if you have - some underlying guess about what's going on, so you can choose tests that show if you're right. If you have no idea what's going on then tests aren't any more valuable than the random numbers."

"Yeah, that's basically it.  In terms of what you should be feeling using Probability-Sight, it's that you've got to extend out the predictions, the 'likelihood-functions', from the things that might be true, to possible observations, so you can imagine what sorts of observations might distinguish between things that might be true, and then steer into that region of reality so you get to see those observations."

"In terms of underlying Law, there's a huge amount of math here of 'computer-science', about what determines how many observations you need, in principle, to narrow down between theories in particular classes.  None of which is a practical priority for you to learn right now, because mortals are so enormously less efficient than gods, or so I expect, who in turn are enormously less efficient than 'ideal-agents', so the principled bounds hardly matter, you're going to be buying way way more evidence than a god would need to arrive at the same conclusion.  But among the things you learn from studying those impossible ideals, is that it takes much much less evidence to figure something out, when you're allowed to pick which questions to ask.  It's often the difference, not so much between 'can be done with ten copper' and 'can be done with a million gold', as 'can be done at all' versus 'can't be done at all'."

"Next I ask - is there any wise advantage in looking at a quantity, a 'scalar-quantity' like the amount of time I take to answer?  Why the amount of time, instead of, say, my facial expression and where my eyes were moving?  Not that I was letting my eyes move down the page in order, or anything that easy to catch, but still.  Why focus on the amount of time I took, which collapses down to a 'scalar-quantity', and not, say, the order in which I approached students when more than one student had their hand raised?  Wouldn't the order in which I picked students be a more complicated, structured, interesting thing to try to predict?  Is there any advantage to trying to measure 'scalar-quantities' besides that they could be measured more precisely, using a pocketwatch that I didn't catch anybody actually looking at?"

Asking dumb question is better than asking no questions. And so he will let himself be scourged by failure, again. "Even without a pocketwatch, the difference between five seconds and six seconds is noticeable. You can compare questions that received five-second waits to those that received six-second waits, and notice that there are gradual increases in the wait over time... I think that latter is important. You cannot notice a gradual effect changing over time in the order you approach students - without a truly tremendous number of tests, that is. You can in the time. But I do not know how to put that into the language of probability-sight."

"Wrong!  But positive feedback for being courageously wrong out loud instead of quietly clutching your error to yourself and leaving me, your teacher, in a silent void of zero evidence about what any of my students could possibly be thinking!  I'm also a kind of thing that requires observations to feed on in order to know things!  Lots of them, even, since I'm not a god!  Yet!"

"There's all sorts of phenomena in the ordering that you could potentially notice right away, if you knew to look for them, and that they were there.  For instance, maybe after you choose a sequence with any big number, I approach you last out of all students currently with their hand raised, and after you choose a sequence of all small numbers, I approach you first."

"But even if that had been true, and somehow important - like big numbers being harder for me to calculate and therefore unpleasant and so I'd subconsciously started avoiding people in the middle of testing those hypotheses - it would have been a harder pattern to notice in the ordering data, compared to noticing how sometimes a number is big and sometimes small.  Actually, even the example I gave of a pattern is really something that got projected down to a scalar-quantity - on a scale of beginning to end, where does my selection of you land inside the set of people with hands raised?"

"If it had been anything more subtle than that - maybe, like, my approaching a new candidate rather than current researcher next, if you showed me any big number - then while it might be very easy to verify that was going on, in terms of updating probabilities, once you noticed anything interesting at all and came up with that hypothesis, that pattern would be a lot harder to notice initially and pretheoretically, compared to noticing a number getting bigger or smaller."

"This is both a useful fact and a cautionary one.  Measuring things in scalar-quantities can give us results that are, in a certain sense, easier to work with - easier to try to notice things about - but that we find it easier to work with scalar-quantities is a fact about us, not a fact about the things.  If you're not measuring the right scalar-quantity, or if the critical measurement isn't a scalar-quantity at all, then you're just - uh, dath ilani proverb.  Somebody loses... a small precious object... a platinum piece, something worth their time to search for... while walking outside at night.  While retracing your path, it might make sense at first to go looking in any well-lighted places along which you'd walked, just in case your platinum piece happened to be there."

"But if you don't find it there, the first time, you need to go get a 'flashlight', a light-generating object, and go looking along the rest of your dark path where it isn't as convenient to look.  It's a comical character, a silly-child character, who insists that they'll go on searching the well-lighted areas because those are the easiest places to look."

"If what you need to watch is in fact my facial expression and not how long I take to answer - then you'd better not get too attached to measuring things in scalar-quantities.  But scalar-quantities sure do make sense as a thing to quickly check out first, if you have a choice of things to look at.  In fact, you might find it so much easier to notice patterns there, that your Science-mind gets out of the habit of even looking at things that aren't easily-measurable scalar-quantities!"

"With that caution in mind - scalar-quantities are easily related to each other.  Not just that, easily related to each other by math, which makes it easier to notice all sorts of mathematical patterns in them."

"If you'd started recording my times more exactly, and found that my average delay times were growing over time for every student, but growing at different rates for different students, you could start looking for other scalar-quantities that were growing at different rates for different students.  Then one of the scalar-quantities you checked might be 'the number of previous inputs that student had already written down'.  Then when you found that my delay time was roughly and on average proportional to the number of previous inputs - in a way that was similar across all the students, even as different students had different numbers of previous inputs growing over time - you'd have a critical insight into the structure of the Keltham-environment's behavior."

"A habit of measuring all the even slightly relevant-looking things that come out in scalar-quantities, or in Baseline, 'quantifying' the 'quantifiable-observables', is something that Civilization has in fact found incredibly useful.  When we start messing around with spellsilver, we're not going to be trying a bunch of random crap to see if anything works, even as written in the language of 'chemistry' where random tries are much more likely to do interesting things.  We'll crush a lot of spellsilver ore down to sand, and mix it all thoroughly so every batch of ore-sand we use is almost exactly the same across experiments.  We'll try small perturbations to the process, most of which will be harmful, but we'll quantify exactly exactly how much less spellsilver came out the other side, and compete to come up with theories that try to call those experimental results in advance."

"Even if most things we find make the process worse, those are still facts we can use - especially in combination with other dath ilani knowledge - to build up 'testable-hypotheses' about what's going on inside the spellsilver-refining process."

"Am I saying the same thing as you if I say 'it's good to measure lots of things, but it's especially valuable to measure things you can turn into numbers on a common scale, because those are some of the easiest for humans to notice patterns from? So when stuck, measuring your response times was likelier to be useful than noting whether you blinked or not?"

"Yup!  Well done at simplifying back into Taldane!  Though, to be clear, it's even more easier for machines to notice patterns in numbers, but you won't have any number-noticing machines for probably a fair while."

"But all of that, this entire previous discussion, is something you could also say to an individual trying to understand, analyze, and improve spellsilver refining."

"What distinguishes 'SCIENCE!' from merely ordinary individual Lawful experimentation with the goal of probing for unexpected patterns or distinguishing among hypotheses already formed, is that 'SCIENCE!' is a protocol for stacking the efforts of multiple individuals - the efforts, indeed, of all Civilization.  It's a designed artifact-shadow of Coordination in much the same way as Governance."

"What, do you suppose, would be the key ingredients of 'SCIENCE!' as a multi-agent practice of Civilization?"

"They get the smartest people to organize it?"

"They offer rewards for correct theories." And punishments for incorrect ones. 

"Standardization of weights, terms and measures, and requiring universal and public record-keeping." Because otherwise people could pretend to lose less 2s than they, in fact, lost, by selectively reporting only the predictions they did well on.

"They do it with prediction markets."  Because he explained those shortly before explaining this.

"You need either a way for people to see and interpret the results of other peoples' individual experiments, like standard reporting of results, or a way to aggregate individual guesses into an overall best guess without anyone having a reason to say something other than their real best guess - if you have that, you don't need everyone reporting their experiments, just betting on reality with the knowledge their experiment gave them. Prediction markets do that, and also do another thing you need, which is a reason for people to go out and do experiments instead of staying home and making their numbers up."

"Prediction markets actually don't work well enough at paying people to go out and secretly do experiments.  The market traders are pretty good at guessing when somebody else is just coming in with new information, and the old traders won't sell at their previous prices - the price moves sharply enough that the experimenter often can't make much of a profit off the prediction market.  Like, the current price is 80%, but that doesn't mean you can just buy a million shares at 80%, reveal your experimental results, and resell at 90%.  Once you start buying in massive quantities like that, the price starts moving upward from 80%, because market participants have guessed you have private information."

"It's more straightforward and direct and less clever than that, actually; what happens is that people will just straight-up promise 'impact-purchases' on discoveries they want to see made."

"Like most dath ilani males, I am 'gynosexual' and very 'straight', meaning that I'm only attracted to women, and mostly only if they present with fairly central feminine gendertropes.  Like most such men, I committed to the outstanding bounty for finding a reasonably-completely-safe 'medical-enhancement' intervention that would make me also be attracted to men, because, as all the 'bisexuals' keep telling the rest of us, we're missing out on half the fun.  The collective commitment is up to something like a hundred billion unskilled-labor-hours at this point... about five hundred million gold pieces in Golarion terms, maybe possibly?  That's how rich you would be in dath ilan, if you singlehandedly found a safe way to turn men bisexual, all by yourself with nobody else contributing.  It's one of the biggest, most famous unclaimed bounties."

What.

(No comment, this is a class.)

- most men, if their battalion gets deployed somewhere where there aren't any hangers-on around to keep them company, like at the Worldwound, find they're actually into men after all. Or, you know, able to get a blowjob and pretend it's a girl at absolute minimum.

Carissa is not sure this is a valuable observation.

"Is this in fact a problem you, personally, want to solve."

"It's relatively less important to me now that I have otherwise saturated dating opportunities, but that doesn't mean I'm not curious."

"...I think it would legitimately be way way down on my list of priorities, given the current competition for my priorities list.  Why?"

"I know some categorically adequate sex therapists who'd probably take you on out of sheer curiosity. People in Cheliax have also encountered this problem, usually the other way. It does seem like a stupid use of your time but who am I to discourage you on those grounds."

"How do those work - is there a spell for it, or a magic-item for that matter?  It's a less stupid use of my time if it's less time."

"You drink potions and have a great deal of unusual sex that expands your horizons. I haven't undergone it, if you want details I'll have to call on a friend. And then murder her, I guess."

- would alter!Lady Eulàlia -

Yes, absolutely, Chosen of Asmodeus. If anything Taldor's nobles are worse.

"I'm going to assume that was a joke even though I can't tell which of, like, three different possible jokes it was.  Specifically, are you supposed to be too honest to just not tell your friend why you're asking, too carefree to care about her putative discarnation, or just signaling the lengths you're willing to go for 'security-mindset'?  Not actually important, don't answer right now." 

"Anyways, I think my current sexual horizons are expanding fast enough, so I will table this issue for probably several months, unless somebody turns up who'd be an incredibly perfect match for me if only I were attracted to men."

"You were telling us about Science," says Gregoria.

"I was, and in particular about how Science! gets paid-for by Civilization collectively.  Something like the long-standing Quest for Safe Bisexuality Enhancement is not likely to get completed by one person; even the whole corporation that invents it will be building off the work of hundreds of other researchers.  The commitment I made doesn't get paid out just to whatever corporation invents the treatment; it goes to one of several extremely reputable philanthropic-redistribution groups that would, if Safe Bisexuality Enhancement got invented, pay out to purchase the 'impact' of whoever had turned out to contribute the most to getting it done."

"With a genuinely massive bounty like that, there's multiple 'venture-philanthropists' who are playing a multi-decade game of funding promising scientists to work on related investigations in exchange for a portion of their 'impact', and then reselling their shares of 'impact' of people who made discoveries that will plausibly be worth hundreds of thousands of gold pieces later, if Safe Bisexuality Enhancement gets invented with their work having contributed 0.1% of what got done."

"That infrastructure is too sophisticated for Golarion in its present state.  The key point is that, like everything else in life, if you want good Science! you've got to pay for it.  'What you're not willing to pay real money for, you shouldn't complain you didn't get', as the proverb goes out of dath ilan."

"What we'll do on the Project instead is as follows:  If you come up with a truly unique and brilliant idea for refining spellsilver, and, this is the part I worry may end up generating 'drama' - emotional-fraughtness - it's not a brilliant idea that I strongly expect I or anybody else would've come up with anyways given another thirty minutes, you get a larger share of the Project."

"Where the issue here is that, especially once you're speaking the correct language of 'chemistry', there are going to be useful ideas that seem brilliant and that, in fact, anybody else who competently thinks in 'chemistry' language would probably also come up with given slightly more time, and we can't afford to give away 1% shares of the Project every time somebody has one of those ideas."

"I don't have anything better to do about that than appoint myself final judge, and go under truthspell on request to say that I'm being honest and impartial about it.  Tap myself with the Fair Pricing spell, too, in case that also helps show the absence of rationalization and self-serving reasoning.  In Civilization there'd be people who spend their whole adult work-lives just specializing in that one form of judgment."

" - this is in addition to, uh, the salary and project-shares already discussed?"

"The shares already discussed are based on your expected contribution, as expected by me at that time.  It includes some amount of unique contributions, because everyone on the Project is expected to make some unique contributions."

"But Asmodia, for example, is angling to clearly contribute more than the other tier-1 researchers and more than I expected at the time I made my hiring offer.  If she ends up much better able than anyone else to teach Law to other researchers in my place, she can in fact pull that off."

"Similarly, if one person on the Project proves to be the only person in the first year able to master 'chemical' Prestidigitation, and ends up spending a lot of their time tediously overseeing the refinement of thousands of pounds of spellsilver, that's likewise a bigger and more unique contribution than I originally expected when I offered them 0.1% or 0.2% of the Project.  You can afford to give 2% to everyone like that."

She could really stand for this project to involve less openly talking about money. It's so gauche.

"Going back to Science.  In answer to the suggestion about getting the smartest people to organize Science! - that follows automatically from offering huge payouts for discoveries that people are sufficiently interested in.  Smart people, being smart, will go where the money is; why would they want to be paid less per amount of effort?  Even Good smart people will usually go where the money is, because that's what Civilization is saying it cares about."

"Also to be clear, if you discover something important that nobody even thought to ask for or expect, plenty of 'utility-buyers' will show up and pay for the 'impact' on that, too; Civilization has done that many times before, because it's important to be consistent and predictable about that sort of thing.  I'll be aspiring to do the same on this Project, which is all that Golarion has of Science! for a time."

"Standardization of measures... I'm not sure what it would mean for something to be a 'measure' that wasn't 'standardized', but, this being Golarion, I'm afraid to ask.  Yes, if you measure something in dath ilan, you would report on that in the same units as everyone else in Civilization uses.  Otherwise people wouldn't build on your results, and that would decrease the amount of 'impact' you got paid for."

Surely there are also smart people who don't particularly care about money. Or maybe she's just too stupid to understand why being smart enough to do Science necessarily entails also caring about money.

- wait, actually that makes sense, if Science is specifically about wanting to understand the universe so you can conquer it. So if you're like Korva, and you don't care about that, then you won't be any good at Science in the first place.

Ugh.

"So I'm noticing that I'm sort of bouncing off the question of how to do Science! exactly, because it's too large, and not really the sort of thing that gets taught to dath ilani children in a single organized lesson, we take the pieces for granted -"

"One of those pieces is legit the part where contributors get paid for doing it, and people who make big contributions to Civilization that way get paid a lot.  I'll be improvising that for the Project as it goes, not ignoring it, and everyone should know that.  Obviously whatever Science! you get will be whatever kind of Science! you pay contributors for and incentivize.  Okay, that said, move on."

"Peranza suggested that Science! would use prediction markets.  That's true, but an easy guess, because every part of Civilization uses prediction markets for everything unless there's a specific reason not to.  Where exactly would Science! use prediction markets, Peranza?"

"What seems obvious is, to predict which experiments would work?"

(AlterPeranza is less terrified of being called upon by Keltham like that, of having it exposed that she was just wildly guessing based on what Keltham calls the 'meta-game'; she is able to quickly think of a plausible thing Peranza could have been thinking.)

"What does it mean for an experiment to work?  You get results.  The results are Reality.  In what sense could Reality 'work' or 'not work'?"

"Well, in the most recent case, 'works' is getting to your first NO instead of all the YESes.  You could have a prediction market on which tries would do that successfully."

"That's a dangerous way to start thinking - one we're explicitly warned against as kids.  You don't say that some results Reality can give you are the experiment 'working' and some are 'not working'.  Think of the timing measurements.  Willa's and Alexandre's first observations there didn't get them 'NO' answers, but those were far from failures."

"That's the same mindset you'd have to break out of to solve spellsilver refining.  Experiments on spellsilver don't have failed results, or successful results, they just have results.  Every experiment tells you something truthful about Reality, which is the world you are embedded in, it never lies.  Maybe your instruments are broken or don't do what you think they do, and then what Reality is really telling you is not what you think you are hearing.  But that's your own error.  Reality itself never lies."

"With that said, Civilization is obviously running prediction markets all of the time on what some particular experiment will say, in its results, or what a hypothetical future experiment will say.  Prediction markets are a fundamental part of doing community experimentation, because prediction markets are a fundamental part of collective 'epistemology' - the way that a Civilization can be said to 'believe' anything apart from the beliefs of the individual people in it."

"For example.  What would it mean to say that an experiment had a 'surprising' result, if there's no prediction market assigning that result a low probability?  Maybe it surprised you personally, but then maybe you were just being stupid and not seeing what others would consider obvious.  Why should Civilization care, if just you personally were surprised?  Why should an 'impact-buyer' believe you, if you claim the results would have been a surprise to society?"

"Or let's say that you make what seems to you like a really incredible discovery, a way to refine ten pounds of spellsilver ore into twenty pounds of spellsilver.  Call the newspapers!  The newspapers are... somewhat skeptical about this incredible claim?  If there's a prediction market, they can check that prediction market to see what Civilization thinks will be the result of duplicating your experimental procedure.  If there's no prediction market, the newspaper just has no idea what to believe and it sounds weird, or so I imagine, and so your possibly incredible discovery drops into the void."

"Or maybe I'm too optimistic about how it works in Golarion, and instead the newspapers print excited stories about your spellsilver discovery, which any real spellsilver scientist knows has to almost be certainly false, but Intelligence 10 people read the newspapers and now they think that's how spellsilver mining totally works and they're confused about why nobody else is adopting this great idea.  Actually that's still too optimistic, actually they'd just stop believing things they read in newspapers, including the things that experts actually did believe.  Or maybe that's too optimistic and -"

"Anyways!  Golarion having no prediction markets, is perhaps alone an 'already-sufficient-explanation' of why Golarion has no Science! and is so far below dath ilan's level of knowledge.  Nobody knows, now, at which point in history Science! was invented - at what point pre-Civilization really started learning collectively rather than as a set of individual experimenters working in isolation or small groups or factions - but I would expect that it only happened after the simpler idea of prediction markets got invented first.  You have to be able to say what Civilization believes, before you can observe or measure how fast Civilization is learning, or figure out how to make it learn faster."

Checking with Asmodia - in alter Cheliax like in real Cheliax are the newspapers forbidden from publishing sensationalist nonsense? 

Snap decision - in alterCheliax it's illegal for a broadsheet to say that the Church or Crown affirms something they don't affirm, and that cuts down on the worst of the nonsense among relatively intelligent people.  But it's not illegal to be wrong, so long as nobody gets their person or reputation injured as a result, and nothing being said is treasonous to the Crown or insulting to the Church.  Sevar, that sound right?

They're going to have to produce broadsheets consistent with it, possibly immediately, but sounds about right in the sense that going more dath ilani than that would be uncharacteristic.

Oh.  Right.  They have to be able to produce the broadsheets on demand, if Keltham insists that somebody Teleport right out and get one right away, as a Conspiracy check.  Asmodia should have thought of that.  And they have to look like Taldorian broadsheets, not Chelish ones like people in Cheliax already know how to produce...

...this sounds like a nightmare for the Conspiracy, actually.  Maybe broadsheets were found to produce enough misinformation and riots that they had to be allowed only to a few Crown-supervised offices?  It doesn't make Cheliax look like Absalom, but it would explain why alterCheliax's broadsheets are relatively small and look government-produced.

The alternative would be grabbing some Taldorian broadsheets and rapidly retraining some Crown authors to be able to produce more of the same, with most of the content ready to go and print as soon as Keltham demands it for any particular day.

Ione or Meritxell, comment something that doesn't have anything to do with newspapers?

Korva Tallandria has a newspaper-unrelated question ready, that she wasn't sure whether to ask, Security can prompt that one.

- oh shit, someone is listening, okay -

"Does that mean that political unification and a lack of factionalism are also necessary prerequisites for Science?"

"Not sure I'm following the logic behind the question?  You need a single market for predictions, or rather, you'd expect to end up with as many separate Science!s as there are separate markets.  But a single worldwide market follows automatically from 'arbitrage' - if one market is buying a proposition at 80% and one market is selling at 60%, you can buy in the 60% market and sell in the 80% market and make a guaranteed profit and do that until the prices equalize."

Why did people tell her to ask it if it was a dumb question!!

"Well, uh, I guess I was thinking that if different countries are doing their Science secretly, then that's sort of like what you were contrasting it with, the experimentation of a small group or a faction, and I was wondering whether a worldwide effort was a necessary part of the concept. I guess you might not need political unification for a worldwide effort like that, at least on some things, but I don't think it happens automatically - the markets would have to know about each other, both in terms of not being secret and in terms of having the ability to reliably trade information even about things that aren't secret."

"Oh, sure, there's all kinds of secret research in Civilization.  Startups working on something proprietary, probably some stuff Governance does that would be socially-infohazardous like scalable-weapons research, and, one assumes, almost everything to do with the Keepers."

"That's just, like, mini-Science! where you pretend to be your own little Civilization.  Obviously if you could have one big Civilization under the Law of Coordination you could also have several smaller ones being governed by the same rules.  The only difference I can see is that any smaller mini-Science! like that would be 'shoulder-standing' the bigger Science! project of the rest of Civilization of which it's part."

Relayed to Korva:  We needed a question for Keltham that wasn't about 'newspapers' while we tried to figure out how broadsheets worked in alterCheliax.  Your sacrifice and obedience is acknowledged.

Oh that makes sense. She is glad to have been of service to the project, and surprised and grateful for the context. She's going to nod and shut up now.

"Okay, what else is part of Science!... replication and reproducibility, right, you'd think that would be obvious and emergent-from the rest of the setup, but it got taught to me as a distinct principle so somebody thought it needed separate focus and emphasis."

"If in Civilization you report that some procedure works for refining ten pounds of spellsilver ore into twenty pounds of spellsilver, the prediction markets on the result of a general or generic experiment like that one, don't all immediately go to 100 and pay out.  In this case, even if you took 'video recordings' of your experiment so people knew you weren't just lying - that you had, at least, put in a lot of effort to fake the video - market traders would still be pretty skeptical.  The relevant prediction market would be on what would happen if known, competent, careful, previously validated, third-party professional 'replicators' tried the same experiment."

"And I doubt that prediction market would go very far above 0%, if the claim was that you could refine ten pounds of ore into twenty pounds of spellsilver, assuming spellsilver was otherwise known to be mostly pure metal and not have components that could be drawn from the air or other refining materials."

"Now, if you're the original discoverer, you probably find that pretty annoying, right?  You published your experimental results, and those stinky prediction markets still don't believe you?  So after double-checking your own work - at least, you double-check it first if you're at all sane or smart -"

"Well, mostly, you discover that enormous blatant error you made, where the scales you were using to measure the ore weight were broken."

"But let's say that doesn't happen.  Then you buy up all those prediction-market shares that are trading at under 1% that your experiment replicates.  The market price starts to go up, obviously, but as it starts to get above 2%, more traders start to come in, checking to see if they think they can make a quick 2% profit off you.  Your prediction sounds completely nuts, they figure this is just a prank or somebody making a weird point, so they start to sell at 2%."

"You buy more.  Some of the less confident counter-traders drop out, when they see you're willing to spend that much."

"Sometimes the way the story goes from here is that the probability gets as high as 20%.  If the proposition at stake is an important one, 20% is high enough that an outside 'venture philanthropist' will come in and fund a 'replication' of the research done by professional 'replicators', because the 'impact' of 'replicating' the original maniacal-experiment, if the results 'reproduce', will sell for more than five times the cost of doing the 'replication'."

"Let's say that doesn't happen, because the idea you can refine ten pounds of ore to twenty pounds of spellsilver is such a ridiculous one that your counterparties are still confident.  They keep buying the prediction's price down even as you keep trying to buy it up.  It never gets above 4%."

"Once you've spent a large portion of your savings on amassing a huge position like that, you spend a pretty large chunk of whatever's left, on the actually quite expensive project of paying respected 'replicators' with the 'certs' to actually resolve the prediction market, to test your method.  Which they will do very very carefully, taking videos of everything, that you inspect to say beforehand rather than afterwards if you think they're doing anything wrong along the way."

"Usually the way this story ends is that somebody loses a chunk of their savings and gains a valuable life lesson about overconfidence."

"Sometimes, very rarely, it ends with a lot of shocked prediction-market traders who lost a much bigger collective chunk of their own wealth, and a story that makes all the newspapers.  Your name becomes one of the glorious Science!-heroic stories that inspires the next generation to not just believe what everybody else believes.  World-class professional sex workers will compete to seduce you just to be able to advertise to their other clients that they won that competition."

"Also you can sell your prediction market shares at a 25-fold profit.  But that's not why you did it, really.  The money is just how - how other people can know that it was all real, because real money changed hands."

... "Where does the typical researcher get the funds to pay for all these 'prediction-market shares' and 'replications'?" It's not like researchers are wizard with spells to sell, right; dath ilan doesn't have wizards.

"Well, if you are ten years old, and your parents aren't very wealthy and indulgent, and your theory isn't so persuasive that you could convince other 'venture scientists' to go in with you on it, then you are not in fact going to pull this off."

"That's considered a 'feature' of Civilization, not a 'bug'.  It prevents Civilization from being snowed under with futile replication attempts for silly ideas.  You can be poor and persuasive and still get an expensive experiment performed, or rich and unpersuasive, but you've got to be at least one of rich and persuasive."

"That said, let's say that the prestigious replicators earn four times what you do per workday, and it'll take five of them working for ten days to replicate your experiment, plus some relatively lower costs to rent equipment.  Then their cost is on the order of two hundred days of your salary.  If you save a third of what you earn for five years, you can pay for one experiment like that and still have two-thirds of your savings left over."

"Also to be clear, if an experimental report is being put forward by anyone who's not a ten-year-old, there will always be some prediction market about whether a 'replication' would 'reproduce' those results.  You're expected to state your initial price and subsidize that market at least a little, which is how you say to the world that you actually believe your own results at all.  But Civilization knowing more stuff, is to some irreducible degree a 'non-rival non-excludable public good', so the 'philanthropists' come in and subsidize a process where some experiments will be randomly selected for 'replication'.  That is, the selection is random from a probability distribution that goes according to complicated formulas that include the market odds, the 'market-volume', and any advance-obvious 'impact'."

"The point being, almost any experiment has some probability of being actually 'replicated', in which case the conditional prediction market on the outcome of the 'replication', will pay out.  That incentivizes accurate prediction markets and careful trading, even on relatively minor experiments that wouldn't usually pay to get 'replicated' by one of the big expensive high-trust professional 'replication' firms."

"Or as my teachers emphasized to me a lot - if an experiment didn't have any prediction market price on 'reproducing', how would you have any idea what Civilization thought of it?  Would there really be a sense in which Civilization collectively knew anything about the result?  There being a prediction market, with some chance of the replication actually being performed in order to incentivize careful trading, is a key condition for an experiment really being part of Science! at all.  Without that, maybe somebody knows something about that result, but Civilization doesn't know anything about it."

"Probably the people who actually think about Science! in tremendous detail are betting that, if we stop making prediction markets for everything, Science! will start to fall apart.  So that part gets emphasized to children, to prevent that scenario?  I don't know the details, unfortunately, I'm just guessing based on how much different stuff got emphasized to me."

"Are there - other known Science failure modes, ways all of Civilization stays wrong about something?"

"So another precaution they emphasized was 'preregistration', you describe the experiment before you perform it and open a prediction market on the results before they're in.  That forces Civilization to make its own prediction in advance.  It lets anybody object beforehand instead of afterwards, if they have an issue with your experimental procedure.  It makes them look less persuasive if they wait until you get a result they don't like, to claim something was wrong with your methodology.  On a Civilizational level, we were told that 'preregistration' guards against a scenario where people think that boring or expected results are 'failures' and don't bother to report on those, until somebody by random luck gets a misleading exciting result and reports on that."

"There's a whole dramatic debate about whether or not, in principle, we ought to pay any attention to results that aren't 'preregistered', if somebody reports on them anyways.  Because of how, if you say we should throw the results like that away, it means we're virtuously refusing to update on what is in fact evidence.  Conversely, if we pay attention to results like that, we're creating bad systemic incentives."

"Obviously, that scenario basically never comes up in real life.  To the extent it ever did come up in real life, I figure people would just toss the original result and pay a 'replicator' to do it over again, if it was anything important."

"But, uh, I'm realizing as I say it, that none of this stuff is what we need to think about on a mini-Science! project the size of Project Lawful.  We're small enough that we can just know about all the experiments everyone is doing, and say all our individual probabilities on them out loud in advance.  We don't have professional replicators to appeal to, all we can do basically is run any important experiments twice with a different researcher in charge."

"I'm sort of flailing here, going through things in random order, as you can probably tell.  In dath ilan we all sort of grow up knowing how Science! is supposed to work, if only vaguely to start with, so at no point does anybody try to teach you the whole thing all at once.  There's just little pieces here and there where you learn in more detail how some bit of Science! works."

"What else is there to doing Science!... I suppose there's integrating the results from multiple experiments, reporting them in a unified way so that multiple lines of evidence can be collected?  Somebody mentioned that part already?  That part is sort of obvious if you know any Law of Probability, there's only one obviously correct way to do that...  Well, no, there was that one kid in class who invented his own wacky way of doing it, so it's not that obvious.  Probably I need to get out of the mindset of believing that saying obvious things will bore everyone here.  Golarion isn't doing it yet, so it's not that obvious."

"Though collecting results across multiple experiments is also something you'd do just as an individual investigating something on your own, meaning it's not really Science! as such... but you know, whatever, it'll be actually useful to the Project.  I'm just going to talk about accumulating data across different people's experiments, whether or not you call that by the term 'Science!'"

"Or no, let's go ahead and maniacally experiment to determine whether it actually is obvious or I just grew up that way.  Asmodia, how would you collect together the results from three experiments meant to distinguish among three hypotheses?"

"Keltham, you literally ran that exact problem on me earlier, when you were 'trolling' me, remember?  Luckily for your real question, I'd worked it out on my own before then, while wearing the artifact headband, and I can confirm that it's obvious."

"But yeah, say you've got three hypotheses, like, 'Ordinary Asmodia', 'Conspiracy Asmodia', and 'Time-Traveling Asmodia'.  They all start out with some probability-weights attached to them - Keltham didn't actually say what his prior weights were, but let's say they were 0.90 Ordinary, 0.06 Conspiracy, 0.03 Time-Travel, and 0.01 for Something Else but we're not going to consider that part -"

"Asmodia, I was supposed to be dead forever and found myself in another dimension with 'alternatephysics' that is both 'economicmagic' and 'conceptualmagic' like the story isn't even trying to be respectable about considering their consequences separately, was immediately allocated a research harem containing masochists, then the god of mind-altering substances cursed one of my researchers to give out cookies and Governance forgot to tell me about that for two days.  I am WELL above 1% on the None-Of-The-Above Hypothesis."

"People in dath ilan are above 1% on the What If Everything About Reality Is Actually Completely Different From How It Looks Hypothesis.  It's just that, usually, there's not much practical you can do about that."

"Uh, 80% Ordinary Asmodia, 10% Conspiracy Asmodia, 3% Time-Traveling Asmodia?  Actually I'm just going to run with that.  Speaking as either Ordinary Asmodia or somebody pretending to be her in great detail, Ordinary Asmodia doesn't want to help Conspiracy Asmodia by asking Keltham to reveal what his probabilities on Conspiracy were at the time."

"Now you're getting it, or realistically pretending to have just now gotten it."

Security, tell some of these idiots to look incredulous or ask 'What?' or something.  AlterCheliax did not have me using this exact case as an Important Lesson in Probability-Law and Also Conspiracy Maintenance and Also Keltham; their alter-personas should be confused about now.

"Um, what?"

"That was probably a little high-context, wasn't it."

"Really?  D'you think?"

"All right, different example.  Suppose there's a murder case with three possible suspects -"

"Asmodia, you can't just not explain now and leave them wondering, that would be mean."

"Keltham is tracking on an ongoing basis the possibility that his new world of Golarion is constructing an elaborate lie around him, including, for example, there not really being any such country as Cheliax.  Do not worry about this or try to do anything proactive about it, Keltham is a dath ilani and will not automatically conclude that everything is a giant lie even if he is looking for possible evidence of that.  We just need to be ourselves, and have faith in his ability to discern reality as reality."

"Keltham was also, I think briefly but I haven't actually asked, tracking the possibility that my overnight personality change and sudden mastery of the Law of Probability was due to my mind traveling a few months backward in time, rather than Manohar dropping an artifact headband on me for two hours."

"Keltham then observed me do two things and fail to do a third.  I forget what his exact probabilities were, at this point," she'll probably still remember them with awful clarity a hundred years later, "but the point is, he - guessed? calculated? I don't understand this part very well - some probabilities for how likely I was to do thing #1, if I was Ordinary or Conspiracy or Time-Traveler.  Then probabilities for my doing thing #2, and then probabilities for my doing #3.  Except I didn't do thing #3, so negate those probabilities - I mean, subtract them from 1..."

"Specifically, I'd just offered Asmodia a lower salary than all of my other researchers, she'd just written something down on a piece of paper, and what I was predicting was, one, the chance that what she'd written was a Prediction about me trolling her, two, if she'd made that Prediction, the chance that she'd assign a probability to her statement, three, if the first two things happened, the chance that she would write down that I would predict her Prediction."

Asmodia goes up to the board and writes the example:

                       Ordinary      Conspiracy    Time-Traveler
                       --------      ----------    --------------
Prior:                   .80            .10            .03

'Writes' (Y)             .50            .20            .80
'Quantifies' (Y)         .60            .70            .90
'Prediction' (N)        1 - .40        1 - .20        1 - .90

"And then - Keltham didn't actually say this part, because it was obvious given what we already learned - he multiplied his starting probabilities by all the likelihoods for each of the events, and ended up with... give me a second..."

Product:                 .14            .011           .002

"Not that it was actually that, because those weren't actually his probabilities, but that's the obvious way to combine multiple pieces of evidence.  Uh, though I'm leaving out the part where what really changes, is the ratio between the chances, not really the chances themselves.  At the end it's like... 13 times as much Ordinary as Conspiracy, and 5 times as much Conspiracy as Time-Traveler, where previously it was 8 times as much Ordinary as Conspiracy and 3 times as much Conspiracy as Time-Traveler."

"If we didn't already have the Law and were looking for Law-fragments, I'd observe that it doesn't matter what order you consider the data in, because it doesn't matter what order you multiply numbers.  Or that you could take the first results and multiply by those and call the result your new 'prior', and then multiply by the second and third results to get the new 'posterior', and that'd also be the same.  Those seem like the kind of 'coherence-constraints' the Law would need to obey, if we didn't know the Law already, or if it wasn't so easy to find a simple Law that fits together like that."

"That's basically correct, except that I didn't actually calculate any prior odds or multiply them?  When you're dealing with weird hypotheses on the order of 'my new world is actually a Conspiracy at a low level of sophistication distinguishable to me from reality', where your own mind is doing a lot of thrashing, it can make more sense for a non-Keeper like myself to make up the 'likelihoods', calculate the combined likelihood, and then just... sort of let my intuitive mind keep track of the intuitive update that it feels after I stare at those likelihoods a bit?  If I was keeping formal track of the chance of Conspiracy, it would probably swing all over the place, because of the degree to which I'm making all of the likelihoods up.  The thing to notice is if it starts to feel to me like there's a trend, after I keep on making up likelihoods."

"But, yes, that's the Law for combining the results from multiple experiments."

Korva doesn't think that she could do that stuff on the board unassisted right now, so she's feeling a little bitter about Asmodia (who is, after all, the only other person in the room qualified to teach classes about alien math concepts right now) declaring it obvious.

It doesn't actually look that complicated, so hopefully Korva will be able to confirm its obviousness and apply it to future things later on, when she's gotten a transcript of this class and her brain isn't dribbling out of her ears. Right now she's busy panicking about all of the rapid-fire baseline words in this lecture that haven't been explained and don't seem to have direct Taldane equivalents, and also trying to figure out what exactly an experiment is. It seems like that might be kind of important to figure out, given that she's apparently not allowed to do any of them without publicly announcing it and giving her predictions in advance.

She's not going to ask this, because a bunch of other people seem to know it already, and if she's in the slower half of the class they're going to kick her out and ensure that her soul is permanently worthless.

"Some additional points, when applying this Law inside of Science!, that might not be obvious... let me actually 'whiteboard' the three points, so I don't forget the later two while talking about the first one."

#1 - 'Published-experimental-reports' usually don't assign 'priors' or calculate 'posteriors', they just report all cheap details of the raw data, and maybe calculate some 'likelihoods' from obvious hypotheses

#2 - Separate experiments are usually supposed to avert 'conditional-dependencies', watch out for when that isn't true

#3 - If every obvious hypothesis has unexpectedly low 'likelihood' over all the combined data, it means the true theory wasn't in your starting set, often that different experiments had different hidden conditions

"Point #1, a 'published-experimental-report' wouldn't make up 'priors' or calculate a 'posterior' the way Asmodia did in her example.   That would be a weird kind of thing to claim as the knowledge of Civilization!  Somebody looking at your report might've seen other evidence.  They could be working on a secret Governance project, and know about evidence you can't access at all.  They might be considering hypotheses you haven't thought of at all.  Or, just, if you don't have formal ways of assigning 'priors', they might have more life experience in guessing those."

"Making claims about what Civilization should believe in the light of an experiment is a prediction market's job, not the job of one experimenter who publishes a report."

"If there's very standard or very obvious particular hypotheses to consider, you might make people's lives easier by precomputing the likelihood of the data on each of those hypotheses.  But one of the big cautions is to consider that other people might have hypotheses you don't, which means you should try to report more details of the data than you already know to be necessary."

"Taking the 2-4-6 example.  Suppose that people just hadn't thought of the possibility that there was 'temporal-dependence' in the results, that they hadn't broken out of the mindset where they were expecting the Keltham-environment to be a timeless 'function' that always produced identical output measurements for identical inputs.  They did run across one duplicate input, but didn't notice it was a duplicate input when they recorded the 'NO' - maybe because two different researchers were querying the environment separately."

"Now think of how inconvenienced you'd be, if that data was reported with input-ordering information missing.  Somebody reading through later, couldn't see which inputs had happened before each other, because you'd implicitly believed that the Keltham-environment was 'time-invariant' and the inputs were 'exchangeable' in the ordering."

"And think of how helpful it would be, if the person had instead happened to include 'timestamps' on each input, even if they didn't know it was important, so you could notice the Keltham-environment taking longer to answer on some inputs, even if they'd missed that themselves."

"If the experiment was on people, the full report includes pseudonyms you can use to contact any of the experimental subjects, just in case somebody comes along later and wants to test subjects to see if they've got a particular 'gene' in their heredity, say."

"Obviously, newspapers can't report all the data of an experiment in the middle of the text.  Even most people who read the 'published-experimental-report' directly won't review all the raw data themselves.  Summaries matter!  So yeah, the report will say, briefly up front and in more detail at the end, 'Here's the theoretical likelihoods of the data for the major hypotheses under consideration.'  The newspaper story will say that, plus, 'Here's how prediction markets on related observables shifted immediately after the results came out.'"

"But the real report, above all, is the data itself - what Reality answered back to you when you asked it a question."

"Or, how did the 'Watcher-over-children' put this... some more of this is coming back to me now..."

"It's not the job of the 'experimentalist' who writes the 'published-experimental-report' to say what Civilization believes, to tell everybody else what to believe."

"It's not the experimentalist's role in society to say what's true."

"That's the job of prediction markets to say."

"Only prediction-market traders get to tell everybody what they should believe.  Because they're the ones who'll lose money if they're wrong.  And if you think you know better, step up and bet yourself, because anybody can."

"So I'd guess there was a failure mode they were worried about, where 'experimentalists' start to take on a social role of saying what's true, and don't lose a lot of money when they're wrong?  Or something like that, anyways.  Though that failure mode itself sounds a bit ill-defined to me, what happens when two experimentalists point in different directions?"

"But yeah, the whole Science! system is put together as carefully as Governance.  Unfortunately I did not pay as much attention to exactly why everything had to be exactly that way, because I wasn't skeptical about the Science! system the same way I was skeptical of Governance as a kid."

Most people, in most parts of the world, probably don't have enough money that it would matter that much, even to them, if they lost it. Not that she's going to point this out.

Keltham's view of a world dominated by prediction-market-traders telling people what to believe is honestly adorable. Would he reconsider, after Asmodeus just was smarter than he is, and won all the bets, and took all his money? Which, after all, would just happen.

"Did this start with - a government doing it, and then doing so well they conquered everywhere else? I don't see how you'd know it was the best way if there aren't other ways it beat out."

"Would you accept 'the prediction markets say the results won't be as good if we don't use prediction markets' as an answer?"

“No.”

"Me neither."

"One answer is that there's always cities inside Civilization trying something different, because, if there is anything dath ilan lacks, it is not people who think they might have Better Ideas.  So far, none of the cities trying something other than prediction markets and traditional Science! have produced vast quantities of new knowledge and technology enabling them to impress the rest of dath ilan into stunned agreement."

"If you mean historically... we don't know a lot of things about our own history that you might otherwise expect us to know, for reasons I'm not getting into at this point.  I'm guessing that dath ilan - had a different trajectory of Intelligence, that something catastrophic happened in Golarion to drive the average Intelligence level down to 10 after writing had been invented at average Intelligence 12 and spellcraft had been invented at average Intelligence 14, or whenever the leading geniuses would be smart enough to do that.  Or writing got carried over from a world that is cousin or common ancestor to dath ilan.  And then something happened to lower the average Intelligence here..."

"My point is, I'm guessing that, by the time prediction markets were around, people generally were smart enough and had enough Law that they wouldn't go to war and conquest, that they'd do something else which was not that, and just, you know, form Civilization, because why not."

"I'm also guessing none of you are going to believe that, and there are obviously other options."

"Maybe prediction markets came along, and Science! came along, and there were factions that refused to adopt that, that didn't know Law, didn't want to be taught Law, didn't want their children to be taught Law.  And then the way of Civilization or what became Civilization, if they thought anything like Civilization thinks now, would be to say - you can refuse to learn the Law yourself, and that's fine, but you cannot choose for your children that they'll have no chance to learn it.  And everybody in your faction is going to have their head 'cryopreserved' upon their death, because to refuse that, is the one mistake that people cannot learn from in time; and you do not pass the competence test to credibly claim to Civilization that you know all the reasons not to do that, and you are choosing to do it anyways."

"And the people in those factions would tell pre-Civilization to go die in a fire.  After which pre-Civilization would say sorry, and come in with superior technology that shrugged off whatever they had in the way of primitive pre-scientific explosives, and teach their children, and save everyone's heads when they died, and plan to apologize about that a thousand years later."

"I - am not sure a world of Kelthams would do exactly that.  I'm not sure we wouldn't, either.  Children are not their parents' stuff, children don't have imaginary ownership-tags pointing to their parents.  I think in the Kelthamverse we'd probably - be less inclined to storm in and do things anyways - if it was about people telling us they didn't want their heads cryopreserved.  Because their heads, their souls, are their own stuff and not ours."

"The children?  Are not anybody's stuff.  If the parents in that dissident faction were, like, not letting their kids own stuff?  The kids there aren't allowed to buy books with Law in them?  I think the Kelthamians probably invade them over that.  All the adults in the Kelthamverse used to be children themselves."

Some parts of those stories are plausible. Some parts are not at all plausible. There are a lot of things she'd like to object to, in that, now that she's thinking about what things are true, but you never object to claims about history, and you super extra double don't object to claims about history with the threat of The Wall hanging over everyone.

...even so, Korva makes a note to ask the project leadership whether they are permitted access to the known information about ancient Azlant, and about the more recent trajectory of standardized wizarding education over the course of the last hundred years, and maybe also about the origins of wizardry itself, if anyone remembers them, and how they relate to other kinds of arcane magic. If that seems like a safe thing to ask for. Obviously.

Asmodia's tempted to just kidnap Korva Tallandria off the Project and put her to work on the Wall, frankly.

This might be a slightly dangerous question, but - "Why is average intelligence important for inventions? For societies, yes -" he cannot imagine the average person in his village functioning in dath ilan "- but how does it contribute to inventing writing, except as it produces inventors of genius?" He has the idea - you could have a broader distribution with more failures and more geniuses or a narrower with more nobodies and what matters for this is the geniuses not the average - he's just not sure how to express it -

"You need both?  There's a lot of work to be done in Science! that doesn't need to be done by geniuses but does take Intelligence 18.  That'd be one of my primary guesses for why Golarion is missing so many dath ilani ideas despite having +6 headbands and artifact headbands.  My other primary guess is that even the artifact headbands are only enhancing some of the key aspects of mentation that go into Science!.  At some later point, we may experiment to see whether Fox's Cunning, Owl's Wisdom, and Eagle's Splendour all together actually help naive subjects on the 2-4-6 challenge, at all.  Though that would be only one of many tests."

Willa's been musing over something in her head, but then she gets a mental approval from Asmodia to just ask about it out loud!

"If your pre-Civilization mostly paid attention to prediction markets when deciding what to do and what to believe, couldn't the factions around it just convince it not to attack them by buying into and distorting their markets? Or to do lots of other things that might not be in its interest? It sounds like a huge vulnerability."

"Do you know what we call it in Civilization when somebody comes in and spends a lot of money to try to shift a prediction market away from the Lawful value?"

"In Baseline:  'Free money!'"

"Prediction markets don't generate money out of nothing.  You can only make as much money off a prediction market by being right, as there is wrong money to bet against you, for you to take.  A ton of wrong money coming into a prediction market is a subsidy that can pay smart people the cost of their time to come in, stare at that market, figure out what the correct price ought to be, and buy up shares trading significantly away from that price."

"From the perspectives of smart traders, it doesn't matter whether the person is foolishly trying to distort the market, or just foolish.  It's a subsidy to them either way.  The more stupid money goes in to be taken, the more smart people crowd around competing to take it at exactly the right price.  People who try to distort prediction markets just end up making them more accurate.  You'd need to be able to literally outbid the rest of Civilization combined, to still be standing there buying shares at a bad price after everyone else in Civilization who wanted free money had run out of money they could use to take your money."

"That's why Civilization trusts prediction markets so much.  They're a form of speech where lying costs you, and then if you try to lie anyways no matter how much that costs you, it just makes the truth stronger."

"To be clear, there is a fundamental vulnerability of this sort, but you can't exploit it just by being rich.  The Hypothetical Corrupted Keepers could prevent Civilization from doing anything they didn't want Civilization to do, by bidding in the prediction markets, openly as the Keepers, to say that dreadful bad results will happen if Civilization does that.  Relatively few people would want to bet against the Keepers - not because it's illegal, to be clear, but because you'd probably lose all your money unless you're Literally Nemamel."

"Then Civilization doesn't do that thing, and the market on that policy never resolves, and the Keepers don't lose any money for lying that way."

"This is a central problem in all of decision theory - that we never get to observe the results of the actions we don't take.  We never get to see our expected utility estimates tested for everything we thought had less than optimal utility.  There's a lot of places where Civilization solves that problem by randomizing slightly, if doing slightly the wrong thing won't be catastrophic.  But the really important predictions - you don't do that there.  And that does open you up to assault by the Hypothetical Corrupted Keepers, or other actors inside Civilization who can afford to put up a lot of money and come up with persuasive arguments and have enough of a reputation to make other traders nervous."

"But it requires acting openly, to make the other traders nervous to bet against your reputation.  There's a limit to how much you can corrupt Civilization's decisions and markets that way, because if you do it a lot, people will be suspicious.  And not just anyone can do it, either."

"...that said, yes, the Keepers have bid against things in inscrutable ways, now and then.  It wouldn't be an especially helpful act to compile a public list of all the times they've done that, but they've done that even in markets I've been tracking.  To this day I have absolutely no idea why the Keepers fear long-term consequences specified to the rest of us only as 'people will later vote that was a bad idea', if Civilization makes a harder push on teaching average kids more 'computer-science' once my generation's kids are slightly smarter.  I mean, it's very credible that 'computer-science' reshapes some people's thoughts in some internally-damaging direction, which the Keepers would rather not point out explicitly for obvious reasons.  It doesn't obviously fit into any plan of corrupt world domination.  But... yeah, what the Keepers bid against, largely doesn't get done, and if they were Hypothetically Corrupted, they could in fact be steering Civilization that way."

Alexandre is now curious what computer-science is. Anything the Keepers don't want him to know has to be powerful.

Asmodia is now curious what computer-science is and if it's going to DESTROY ALL OF GOLARION unless SOMEBODY STOPS IT where SOMEBODY is probably going to be ASMODIA because the rest of Project Lawful is composed of SUICIDAL DISASTER MONKEYS.

The mortal is thinking CORRECTLY again.

It's UNFORTUNATE that this mortal already has an ownership-tag pointing somewhere in Hell, preventing Her from clericing it.  Otolmens has never really understood before what some gods seem to see in their clerics.

Now, however, Otolmens sees.  This mortal would not just be a USEFUL TOOL.

This mortal would be a useful tool requiring MINIMAL ONGOING MAINTENANCE.

"Anyways, I'm going to move onto #2 now, in hopes of eventually traversing this conversational 'data-structure'."

"Point #2, since I was dealing with the same Asmodia over time, I had to consider the 'conditional-dependencies' in my 'evidence' and couldn't treat them as 'approximately-independent'.  I had to ask, assuming that Asmodia was writing down a prediction that I was trolling her, having updated my model of Asmodia off that fact, what chance would she then have of putting a probability on that prediction?  That's not my baseline probability for Asmodia putting probabilities on things, it's the Asmodia who was already predicting well enough to know I was trolling her, which is an Asmodia more tuned in on the rhythm-that-is-Keltham than the Asmodia who didn't do that."

"Then I had to consider that question separately for each 'possible-world' of Ordinary, Conspiracy, Timetravel.  The facts inside a single world can interact among themselves, the facts between worlds don't."

"Uh, not in this example, anyways.  Different possible worlds can start interacting if you get into multi-agent logical decision theory.  Even in this example, Conspiracy Asmodia would be trying to guess what Keltham would think Ordinary Asmodia would do.  But those complications shouldn't come up every day, and in most of Science! we can ignore them."

"In Science!, usually, when the 'replicators' set up a new experiment at a new location, the results there shouldn't be 'causally-entangled' 'to-any-significant-degree' with the results of the previous experiment.  If you find yourself thinking that what you expect given a particular hypothesis, on the new experiment, is being influenced by what happened in the previous experiment, it means you're not narrowing down your hypotheses enough.  Which will hugely complicate any efforts to combine your data, if you've got to consider how all your pieces of data are interacting with each other."

"If I was really making a serious run at updating on Ordinary versus Conspiracy, I'd need to consider multiple possible Conspiracies, drawn finely enough that what I saw in one piece of evidence didn't much affect my likelihoods on other pieces of evidence..."

"Actually, this would probably be a lot clearer with some math."

"So let's do that, considering the hypothetical case of us experimenting to see whether enhancing all three Golarion-known mental stats by +4 would help 'previously-unexposed-subjects' on the 2-4-6 challenge."

"Simplifying things down way too far, for the purposes of keeping the math simple on its very first presentation, we'll say that the only variable we measure is whether or not somebody solves the challenge during their first five minutes."

"Now, how would you do Science! to that, to see whether +4 to all mental stats helped?  I'm going to let researchers and candidates from any tier answer, because I expect this one to be pretty hard just starting out.  But if I'm wrong about that and Asmodia or Carissa are seeing it immediately, do call Prediction rather than just tossing out the complete answer."

She still doesn't know what Science is or what it means to do it to something!!! But - there's an obvious thing to look at, isn't there -

"Well - test whether people who read 14 in every mental stat get it within five minutes more often than people who test 10 in each?"

"With what resources? Strategies will vary depending on what we can afford."

"Yeah, you're going to have some trouble finding people with exactly 14 in all mental stats, I predict, given Cheliax's current state of gathering statistics on its population.  Even if you can find them by checking with all the wizard academies where they measure that, paying to Teleport people in sounds quite expensive."  (The back of Keltham's mind also notes that this proposal is something a not-too-competent Conspiracy Korva might float, if their world secretly had much better stat-gathering and/or cheaper Teleports, and they hadn't fully updated on the fake world not having that fact true; and Conspiracy Alexandre might be trying for a hasty backpedal there.)

"But let's leave that aside for now.  Let's also leave aside that people with all 14s may have differences that aren't just about the mental stats, like, maybe they came from richer families and ate better as children and got better 'nutrition' and that's part of why they're smarter now.  Maybe they have smarter parents and their parents owned more books, or better books."

"Leave all that aside.  We test 5 people with all 10s, and 5 people with all 14s.  At those intelligence levels, five minutes won't cut it, given our previous experience here.  Let's give them thirty.  2 out of 5 people with all 10s get it in thirty minutes, 3 out of 5 people with all 14s get it in thirty minutes."

"What have you learned?  What do you now believe?"

"Let's say on the one hand you guess a 50% getting it if the headbands don't matter. Then this situation has a chance of 0.5^10 of coming up, since each one was a coinspin anyway."

"But if you instead thought 60% with headbands and 40% without, then for the headbanded people it'd be 0.6^3*0.4^2, with the winners being 0.6s and the losers being 0.4s, and then for the non-headbanded people you'd have 0.6^3*0.4^2 too, for the opposite reasons. So that's a chance of 0.6^6*0.4^4. Then we can multiply everything by 10^10 to make the math easier, and the idea of 60/40 and 40/60 increased in probability vs pure 50/50, by, umm..."

"(6^6)*(4^4)/(5^10)"

"Mm, not the Science!tific way of thinking about it, exactly, but not at all bad, either.  You even simplified the math some."

"Though, saying they got artifact-headbanded is not what the original experimental spec was.  We were specifying natural all-14s which will probably give us noticeably different results.  One does want to learn to think precisely about that sort of thing."

"To compute that number there, let's observe it'll be the square of 6^3*4^2/5^5, or 216*16/3125, 2160+1296 is 2160-4+1300=3456, divided by 3125... about 1.1, since 3456 is around 3125+312, squared is about 1.2."

"So the results you saw are 1.2 times more likely if the all-10s have 40% pass rates, and the all-14s have 60% pass rates, compared to everyone having 50% pass rates."

"Now, I realize this ought not to go in a 'published-experimental-report', but what do you happen to believe, seeing that?  What would you believe if you saw that in real life?"

"Oh, yeah, sorry about the headbands I was distracted by the cool math careless."

"But if I saw it in real life, I wouldn't believe very much of anything? It was only five in each group, and two of them randomly going the other way could've made it all look backwards. But times 1.2 isn't a lot of a chance boost either, compared to really knowing stuff about how the world works. So the math seems to look about right? It wasn't too convincing, and the math isn't too convinced?"

"I would go from believing that natural all-14s are going to be at a disadvantage relative to natural all-10s, to believing that having stronger mental stats and having had the experiences that go with them at least isn't actively disadvantageous, and probably does help."

(Oh gods why did she say that that was a terrible idea he's going to make her do math in baseline right now and then everyone will know how stupid she is about this.)

Keltham nods at Korva, then addresses Willa.  "Seems a bit sus that the hypothesis you're comparing to the random-coinspin is one where the supposed natural frequency of thirty-minute guessers, at those two Intelligence levels, exactly matched the data."

"Suppose we'd tested a thousand 'subjects', instead of ten.  245 out of 500 'subjects' with all-10 stats guess within thirty minutes.  256 out of 500 subjects with all-14 stats guess within thirty minutes."

"Then, clearly, to ask how we should react to this data according to your methodology, we should compute the likelihood ratio over the fair coinspin, of the hypothesis that among all-10s, 245/500 guess within thirty minutes, and among all-14s, 256/500 guess:"

(245/500)^245 * (255/500)^255 * (256/500)^256 * (244/500)^244 / (250/500)^1000

"This simplifies to... let's see... a likelihood ratio of about 1.276."

Keltham will now pause dramatically.

"I'll bite, since you're obviously dying for somebody to ask.  How'd you simplify that?  I'm frankly not seeing it."

"Well, if I've managed to remember all the numbers correctly, it's from a version of this problem I worked a while ago, back in Civilization where you could just toss 'expressions' like that into 'computers'*."

(*) This is clearly the same word as appeared in 'computer-science'.

"I see."

"And if you're wondering where the numbers 245 and 256 came from, in that problem, they just happened to be the results of generating 500 random 'bits', twice, and counting 'YES' answers."

"But clearly, the 'computer' could not have been truly randomizing!  After all, it is 1.276 times more likely that these numbers would appear if they had truly been generated by a 'computer' that first picked 'YESes' with 49% probability, and then picked 'YESes' with 51.2% probability.  The Conspiracy has been at work again!"

"Would you agree with this stance, Willa?"

This is more than a little mortifying but at least a bunch of her peers didn't get this right first, she can keep her composure, really she can.

"This is something to do with making the guess about what happened after seeing the data, isn't it? Instead of guessing the ratios first for stats-matter and stats-don't-matter, we did the experiment, and then made up the exact shapes of our guesses after, which is a naughty thing we weren't supposed to do."

"And there's some deep mathematical thing about doing that, looking at data you already have and matching that way, that tends to produce these relative probabilities around 1.25 or so, isn't there?"

"Yup.  Though embarrassingly I don't recall the exact constant.  Back when I was memorizing those numbers in order to impress the younger kids I was teaching, by pretending to calculate it all on the spot, they were too young to ask that question so I didn't memorize the answer."

"But as you say, Willa, you did indeed try to do something naughty, by calculating the likelihoods for 'latent' frequencies that exactly matched the observed frequencies.  Maybe if you'd run a pilot experiment and found 40% YESes for all-10s and 60% YESes for all-14s, you would have written down the numbers 40% and 60% in advance, in the 'preregistered' version of your 'published-experimental-report' that you'd put out and gotten... signed and time-stamped... by larger Civilization, before you'd collected any data.  Then nobody would call you naughty if you ran the likelihoods for the 40%-and-60% hypothesis, and compared that exact hypothesis to the fair-coinspin hypothesis."

"So hopefully it's starting to become clear that all of the pieces of Science! hang together, even if I didn't explain them all that well?"

"Anyways, let's say you didn't run a 'pilot-experiment' that produced 40%-and-60% as a hypothesis distinguished in advance, or 49% and 51.2%.  Then what might you say in your 'preregistered' 'published-experimental-report' that doesn't have the results filled in yet, about how you plan to analyze the data?  People besides Willa are allowed to answer too."

"What are we trying to figure out? We want people who can solve all the problems in 30 minutes? Then we probably care about whether all-14s are enough better than all-10s to make up for how much rarer they are, so we should just check, how likely does it look that all-14s are a hundred times better at this, or even five times better at this?"

"Seems slightly shaded towards 'How do I make more spellsilver?' instead of 'How do I understand what's going on inside of spellsilver refining?'  But leaving that aside, suppose we accept that 'goal-framing'.  What math do we do to it?"

" - huh, so, my complaint is that doing math on how much we should change our minds from a starting assumption there's no difference feels fake. Unless we're doing this study out of idle curiosity it's only a worthwhile study if Civilization was starting with the belief not just that there's a difference but that there's a large one. If 14s aren't wildly better at this then there's no way you'd use them. So the math is easier if you assert that we thought there'd be no difference, but I can't really imagine thinking that. But on the other hand, there's no math you can do on the thing I just said - you would get wildly different answers if you started out thinking 14s are 5 times better versus if you think they're 100 times better - and I thought maybe there was some way to specify 'started out thinking 14s are at least 5 times better' but I'm actually not seeing it. I don 't know if that means there's no math directly for answering the question that would actually motivate my Civilization to do this experiment, or if there is but it's far more complicated than the starting math and we need to get that nailed down first."

"Asking 'what if there's no difference' and 'what if tri-14s are at least five times more likely to guess than tri-10s' are both questions that are semantically well-formed - it's easy to visualize ways reality can be that makes those propositions true or false - but the second question is hard to answer directly using the statistics that 'published-experimental-reports' are supposed to calculate."

"To be clear, you could see a report saying that they tested 1000 subjects of each type, and 103 tri-10s and 622 tri-14s guessed in time.  You could look at that report and say, 'Yep, sure looks to me like tri-14s are at least five times as likely as tri-10s to guess in time.'  But in stating that, you'd be arriving at a 'posterior', not stating the kind of evidence that a 'published-experimental-report' summarizes.  You'd be doing something that intrinsically revolves around 'priors' and not just 'likelihoods'."

"Do any of the candidates want to take a stab at saying why Carissa's question of ultimate interest to her Civilization - 'Are tri-14s at least five times as likely as tri-10s to guess in thirty minutes?' - is something that's ill-formed for an experimental summary to directly summarize an 'evidential-update' about?  Why it can't just report on that the way it could report on a preregistered hypothesis that tri-10s were 40% likely to guess, and tri-14s 60% likely?"

Asmodia, after querying Security somewhat plaintively, sighs and gives the green light to the only new candidate who is reckless enough to risk directly criticizing the ideas of the Chosen of Asmodeus at this stage in their acquaintance.

Can this person please at least look around hesitantly before she tentatively raises her hand?

She very hesitantly justifies this particular incident of recklessness in her mind, both to herself and anyone listening. She just wants to learn things. But she's also very happy she can help the project right now. She does do her best to also look very hesitant, as is in character for alter Willa and also generally appropriate.

"'Five times better at this' seems like a tricky statement to use with this math. When you say a category has a specific chance of its own, you can go through and multiply all the probabilities in the category independently, and compare it to other categories that have been multiplied up independently. But with 'five times better', the two categories sort of depend on each other."

"You could guess that 10s have a 10% chance, so 14s would need a 50% or better chance. Or 10s could have a 15% chance and 14s would need to be better than 75%. Or 10s could have a 25% chance and it would be impossible for 14s to be five times better. And somehow 'are 14s five times better than 10s' needs to take account of all of those at once and everything in between. Or maybe you mean something more complicated than that by 'five times better', but I think that would make this more of an issue, not less of one."

"And it's even worse, because there's an implied 'at LEAST' five times better there. So we need to account for all the imagined frameworks where the multiplier is at least five, all the way up to infinity."

She smiles pleasantly! There are no power dynamics here, they're all just speculating about math! "I agree that all of those make it terribly complicated, and yet - I look at these experimental results and I think 'ah, so 14s, it looks like, aren't better at this by a factor of five.' Clearly in my head I am calculating the odds of observing these results given my theory, and judging them as low; I'm just not doing it precisely. And so what's bothering me is - is it impossible to do precisely, so the thing it feels like I'm doing in my head is an illusion? Or is it possible but hard enough we shouldn't bother without the 'computers' of Keltham's world? Or is there elegant math that makes all those numbers actually possible to calculate?

The reason this is bothering me is - the closest thing I've seen to science is merchant houses, going, does this supplier provide higher quality by enough of a margin to justify their higher prices? They do tests like this all the time. But I don't see how to use this math to answer their question."

"There's obviously ways to do Lawfully anything you can do with your brain by seeing the results yourself.  Sometimes it takes more Law than you have.  Sometimes it involves calculations that wouldn't make sense to put directly into a 'published-experimental-report'.  Still, Willa has put her 'publication-priority-timestamp' squarely on the key issue."

"By way of approaching the complicated proper math by a simpler 'hacky' route, suppose the 'preregistered' analysis said that they were going to compare hypotheses for each 'subject-group' having seven possible 'latent propensities' to solve the puzzle in time, of 0, 1/10, 1/5, 2/5, 1/2, 3/5, 4/5, 9/10, and 1."

"Since some subjects in both groups solved the puzzle, and some didn't, we can cross off the 0 and 1 'propensities' from both cases; they've been falsified outright, the 'likelihood' is zero 'conditional' on those propensities being the true 'latent' state of reality."

"Now, let's go ahead and compute the 'likelihoods' for each surviving 'propensity', in each group..."

All-10s (2 YES, 3 NO): Propensity:     Likelihood: ---------------     -------------- 1/10                (1/10)^2 * (9/10)^3 =   729/100000 2/10                (2/10)^2 * (8/10)^3 = 1024/100000 4/10                (4/10)^2 * (6/10)^3 = 3456/100000 5/10                                               = 3125/100000 6/10                (6/10)^2 * (4/10)^3 = 2304/100000 8/10                                               =   512/100000 9/10                                               =     81/100000 All-14s (3 YES, 2 NO):                                                       = same but table flipped

"The point which Willa is gesturing at, is that it's not possible in the same way to speak of the combined 'likelihood' of all our observations, if all-14s guess more often than all-10s, or if all-14s guess at least five times as often as all-10s."

"At least, not without dragging in some additional assumptions."

Korva doesn't at all see why you can't just ask whether all-14's get it within thirty minutes five times more often than all-10's, do the experiment, and then report whether your records of it confirm or deny that this happened, but Willa and the Chosen of Asmodeus are obviously much better at this than her, so if they're stuck on it then whatever Korva is doing is going to end up being humiliatingly wrong.

"Keltham, advisory, the new candidates have spent their previous educational lives in unimaginably-to-you-awful Golarian schools designed by Intelligence 14 teachers for Intelligence 10 students, where asking stupid questions means that you lose face in front of the other students and the teacher rebukes you.  Right now, most of them are staying quiet and not asking any questions.  This is not the good sign that it would be in dath ilan.  Here, it means that they have made a completely reasonable decision to let other candidates ask stupid questions first, and see whether we - the current researchers - were being honest with them about whether that's safe here."

"I'm not quite sure whether they're internally wondering 'What's wrong with just testing a lot of people and reporting whether all-14s get it five times as often?' or more like 'Why would anybody do this sort of experiment in the first place?' or maybe 'But what's an "experiment" actually?' because the Baseline word translated in my head but not in theirs, but..."

"Thank you, Ione."

"Can I possibly solve this problem by offering anybody here one silver to ask whatever stupid questions they weren't asking?  At least in dath ilan, offering to pay for something is taken as a credible sign that you want it."

"I wouldn't bet on it.  Maybe if you told them to Message me with the questions, let me ask it for them, and then you pay me and I pay them after class."

"What if I literally truthspelled myself about -"

"Same problem as in the cooperation-defection dilemma if you're a Chaotic Neutral outsider.  They wouldn't be certain you would know correctly whether they'd lose face in front of you, and it wouldn't solve the problem of losing face in front of the other students."

"You know, I've read about the concept of 'low-trust environments' but never really understood it before."

"Ione, you okay with being tapped by truthspell about how, if somebody else asks a question through you, you won't reveal who it was?"

"Most of the current researchers will see who, Ione.  We've made our soul arrangements and our arcane sight will detect the Message spell."

"Right."

"Well, I can still re-ask a question if anybody Messages it to me, and at least Keltham and the other candidates won't know who."

"And if that doesn't work, Ione can introduce a delay between being Messaged and when she speaks, and I'll start paying a copper to anybody who pretends to message Ione but doesn't actually subvocalize anything to her, so as to create plausible deniability about who was actually asking questions."

"If that doesn't work, I'll stop and actually think about the problem, though that's something of an extreme measure."

"Seems like a good time to mention that I had that question about 'What's wrong with just testing a bunch of people and reporting whether or not there were at least five times as many all-14s who solved it?'  And didn't ask because I am flawed and imperfect, and forgot the rules about what truly serves Asmodeus here."

"I'm actually not sure how Keltham would answer that one?  It's not thinking in probabilities, though.  And what do you do if you test a few thousand people and 100 all-10s solve it and 499 all-14s solve it, report that the theory failed?"

Korva is sitting on about ten different stupid questions at this point. She would just ask them out loud, at this point, if there was only one of them, but with ten it's hard to pick.

The most recent question is that she thinks she can see what Keltham's doing now - it's something about converting the results of the experiment into probabilities for different guesses about how the world is being the right guess, or something - but she has no idea why the calculations he did were the right ones in order to get those probabilities out the other side.

Ione holds up a hand and seems to listen.  "Okay, Message from somebody who says that their issue is actually that they've got no idea why the calculations you did on the whiteboard are how you convert the results of the experiment into probabilities for how those are the right guesses for how the world is."

Because alterCheliax students would not be this cautious and Asmodia is sick of it and Security is running Detect Thoughts anyways.  This is Cheliax and they have ways of obtaining questions from reluctant askers.

"...huh.  Asmodia, you may have needed to do more basic Probability problems with your students, not just run them through all of the abstract high-level stuff, though I appreciate that you had sharply bounded time."

"Yeah, understood.  I'll see if I can fix my fault there without taking up your own time."

Korva didn't message, so instead of being nervous she's assuming that someone behind her also had one of her questions (not surprising, if Pilar also generated a different one of her questions independently), which makes her feel a little less bad about herself. Maybe she'll at least have another day to go over it and figure out what just happened.

Amusing.  Now the Security wants to see who works it out before they get their silvers after class, and the expressions on their faces if they don't.

Meanwhile, the ever-oblivious Keltham is turning to gesture at his Prestidigitated whiteboard:

All-10s (2 YES, 3 NO): Propensity:     Likelihood: ---------------     -------------- 1/10                (1/10)^2 * (9/10)^3 =   729/100000 2/10                (2/10)^2 * (8/10)^3 = 1024/100000 4/10                (4/10)^2 * (6/10)^3 = 3456/100000 5/10                                               = 3125/100000 6/10                (6/10)^2 * (4/10)^3 = 2304/100000 8/10                                               =   512/100000 9/10                                               =     81/100000 All-14s (3 YES, 2 NO):                                                       = same but table flipped

"Right, so, in the first row, we've got the hypothesis, what if all-10s have a 0.1 'propensity' towards YES."

"Where, in this context, what that means, is:"

"Suppose the world is such a way that a random person from our 'survey-pool', about whom we know only that they have 10s in all mental stats of Intelligence, Wisdom, and Charisma, has a 10% 'propensity' to guess the correct answer on the 2-4-6 challenge within 30 minutes.  Where 'propensity' is sort of a metaphysically fraught concept.  But for our purposes in the kid's version of the lesson, 'suppose 10% YES propensity in all-10s' means, suppose the world itself is such that, from our perspective, it works out to 10% of the all-10s guessing within 30 minutes.  Nethys, maybe, would know that some people in that pool were 99% likely to get it and some were 0.1% likely to get it, so it's not that the 10% chance of YES is an objective property of every single person.  But from our perspective, the world is such a place that Nethys would say that around 10% of the people in that pool were 99% likely to get it, and we don't know which people."

"That's one hypothesis, the one in the first row.  The second row's hypothesis is that all-10s in our survey-pool have a 20% propensity towards YES when we'll test them using our current experimental setup."

"Suppose we test five all-10 subjects - one after another, rather than in a simultaneous session - and see these exact results in order:  NO, YES, YES, NO, NO."

"After getting a NO result with the first all-10 subject, we've seen something that was 90% likely in the world where all-10s have a 10% propensity to YES.  But 80% likely in the world where all-10s have a 20% propensity to produce YES results."

"After seeing the YES from the second subject, we've now seen an additional fact that was 10% likely in the world where all-10s have a 10% YES propensity, and 20% likely in the world with 20% propensity."

"The two facts combined, that we've seen so far, are then 0.9 * 0.1 = 0.09 likely in the 0.1-propensity world, and 0.8*0.2 = 0.16 likely in the 0.2-propensity world."

"We aren't saying anything there, one way or another, about the probability that the world is like that.  We're not saying anything about the probability that Nethys would, if we could ask him or Ione could, write CORRECT on our sheet of paper, if we wrote down the guess, 'all-10s have a 10% propensity to YES'.  To say that we'd need to know the 'prior-probability' of that hypothesis, and the 'prior' on all the other hypotheses, and have already computed the likelihood of all the other hypotheses, and 'renormalized' to get the 'posterior'."

"Which isn't a sort of thing that experimental reports try to do.  Other people could know other evidence that would be relevant to whether Nethys was likely to write CORRECT on the guess.  Stating probabilities like that are what prediction markets are for."

"What we're saying is - just flatly suppose that the world is in utter fact a place where all-10s have a 10% propensity to solve the 2-4-6 challenge within 30 minutes, relative to the experimental procedure we're using.  Then it is valid, as a matter of 'logical-deduction', to say that you're 9% likely to get a NO followed by a YES on the first two subjects tested.  It is likewise valid to say that, flatly assuming the hypothetical world where all-10s have a 20% YES propensity, you have a 16% chance of getting a NO followed by a YES."

"This is the key fact that other people need to know in order to update their beliefs based on your experimental results, so it's what the experimental report summarizes."

"It's a very local fact.  It's like how, if you suppose that X=3 and Y=4, you can calculate that X*Y=12 without worrying about whether Z=5 or Z=7."

"If somebody else already ran tests on 100 subjects with all-10s, they might have seen results that pretty strongly updated them on the chance that Nethys would write 'CORRECT' on the 20%-propensity-hypothesis.  They could've gotten 80 YES results, for example, which would make them pretty sure that hypothesis was wrong."

"But they can't have gotten any results relevant to the proposition that, in the hypothetical world where 20% of all-10s guess within 30 minutes given our experimental procedure, there's a 16% chance that we'll get a NO followed by a YES."

"You don't need to read all of the experimental reports in the world, you don't need to follow any prediction markets, to report that summary of the results you got."

"Suppose your third, fourth, and fifth results are YES, NO, NO."

"After the third result, we've seen data that's 0.9*0.1*0.1 = 0.009 likely in the 10%-propensity world, and that we'd have a 0.8*0.2*0.2 = 0.032 chance of getting in the 20%-propensity world."

"After the fourth result, a NO, we've seen things we're 0.0081 likely to get in the 10% world, and 0.0256 likely to get given 20% propensity."

"After the fifth result, a NO, it's 0.00729 or 729/100000, and... haha, whoops, 0.02048 or 2048/100000 for 20% propensity."

"This, by the way, being among the reasons not to trust your teacher even when he seems like such an uncomplicated straightforward reliable person, it is literally actually possible for him to be mistaken."

Keltham Prestidigitates the corrected table of likelihoods:

All-10s (2 YES, 3 NO): Propensity:     Likelihood: ---------------     -------------- 1/10                (1/10)^2 * (9/10)^3 =   729/100000 2/10                (2/10)^2 * (8/10)^3 = 2048/100000 4/10                (4/10)^2 * (6/10)^3 = 3456/100000 5/10                                               = 3125/100000 6/10                (6/10)^2 * (4/10)^3 = 2304/100000 8/10                                               =   512/100000 9/10                                               =     81/100000 All-14s (3 YES, 2 NO):                                                       = same but table flipped

"I wish I had some way of asking you whether that helped, anonymous question-asker, but maybe communicate again to Ione if that was still unclear?"

Security, don't relay anything further about that topic to Ione unless somebody takes the initiative to ask.  The meaning should be perfectly clear at this point, and if any newbies can't keep up and also can't ask questions, they're obviously headed for the dropout category.

"Nothing yet, which hopefully indicates it was clear.  And if not, well, they were adults and made their own decisions."*

(*)  A saying now acquired from the kidnapped rescued Taldane girls in the secondary site.  Though the same idea certainly does exist in Cheliax; indeed, there exist many Chelish variations on this saying, including several with Infernal loanwords in them and a couple straight from Hell.

Alexandre raises his hand. "I have a question. If I have two hypotheses, say 40% and 60%, I know how to update them based on this data. But at all times -" Keltham has said, Alexandre obviously does not have it at all times "- you have a hypothesis that is - vague, not specific, 'what if all my guesses are wrong'. How do you weight the answer, 'anything but my hypotheses'?"

"Good question!  I'll ask you to hold that thought pending subtopic #3."  Keltham gestures back to his previous list of subtopics he shouldn't forget to talk about.  "Or actually I should maybe just write that one down..."

#1 - 'Published-experimental-reports' usually don't assign 'priors' or calculate 'posteriors', they just report all cheap details of the raw data, and maybe calculate some 'likelihoods' from obvious hypotheses

#2 - Separate experiments are usually supposed to avert 'conditional-dependencies', watch out for when that isn't true

#3 - If every obvious hypothesis has unexpectedly low 'likelihood' over all the combined data, it means the true theory wasn't in your starting set, often that different experiments had different hidden conditions

#4 - How to specially process the special meta-hypothesis 'all-other-hypotheses'

"Oh, and if that last Baseline word isn't translating, maybe the Taldane equivalent would be - everything we haven't thought of explicitly, all the theories we're not considering?"

"Anyways, this has hopefully ended up making #1 a little clearer."

"Going back to #2... what would make a good entrance point..."

"All right, so this isn't addressing #2 right away, just introducing an idea we'll use there, but."

"Suppose that we tried summarizing the hypotheses here into three buckets.  One bucket that the YES-propensity is 0.1 or 0.2 or 0.4, one bucket that the propensity is 0.5, one bucket that the propensity is 0.6 or 0.8 or 0.9."

"Is it then possible to describe the likelihood of our data NO YES YES NO NO, conditional on the first or third bucket?  For the middle bucket it's obviously 3125/100000 or 1/32."

...why did the only candidate who's letting herself generate responses have to be her.

Yes.  But BE FUCKING SHY OKAY.

She makes sure to look back and forth before answering, as if she's making sure nobody else is going to say anything.

"It'd have to be the sum of the likelihoods right, or the whole idea of probability wouldn't make any sense? So 0.1 or 0.2 or 0.4 would be total likelihood of (729+2048+3456)/100,000 = 6233/100,000 vs 0.5 at 3125/100,000"

"Except that only works if you thought 0.1 and 0.2 and 0.4 had the same chance to begin with, before you did the experiment. And even 0.5 too, as likely as all the others. If you thought they all had different before-chances it would mess up the weights and I'm not sure doing that would work anymore. Maybe you'd have to do them individually if you couldn't trust they started off the same?"

"Suppose I've got a coin that might have a 4/10 chance of coming up Queen, or maybe a 6/10 chance of coming up Queen, I'm not really sure, seems about equally likely to be each.  Would you agree that in this case, the coin is (4+6)/10 = 10/10 likely to come up Queen?"

Oh no something's definitely wrong, but she can fix it, she knows she can fix it it's just math...

"No, definitely not, but this is relative likelihoods and not absolute likelihoods? Maybe that doesn't help, hmm"

"Is the idea that we have to divide by the number of cases our bucket used for it to work? And maybe that the before-chance we use then is the sum of the before-chances of all the pieces of the bucket together?"

"Stop trying to think so much in complicated math and fall back on common sense.  I spin one fair coin, 50% chance of Queen, 50% of Text.  If the first coin comes up Queen, I take out a new coin that's biased to have a 60% chance of coming up Queen.  If the first coin comes up Text, I take out a different new coin that's biased to have a 40% chance of coming up Queen."

"Suppose I was actually going to carry out that procedure.  You don't get to observe the prior coinspin.  What's your betting chance that, when I spin whichever new coin I take out, it shows up Queen?"

"It has to be 50%, which is the same as 0.5*0.6 + 0.5*0.4. But if the first coin is in this case the before-chance, we don't know that it's 50/50, do we?"

She has ever gone through cycles like this getting tortured a few times, but being tortured is what happens to good Asmodeans when they're wrong, right?

"I mean, in this case, you know because I told you to assume that setup, and answer me conditional on the assumption."

"To answer the question," drawls Alexandre, looking up from his scribbled notes, "if we assume that 0.1, 0.2, and 0.4 all start with equal prior likelihoods, and similarly for 0.6, 0.8, and 0.9, it's (729/100000 +  2048/100000 + 3456/100000), divided by three because there are three equal possibilities, compared to (2304/100000 + 512/100000 + 81/100000) divided by three for the same reason, gives you -" (he's been doing calculations while Willa talks) "- roughly two thousand seventy over one hundred thousand, as opposed to a little under seven hundred over one hundred thousand, giving you a ratio of a little under three to one in favor of the first bucket of theories against the second bucket."

Wait no that's obviously wrong oh damn it -

"Does that work?" asks Korva, sounding genuinely unsure. "That gives you a lower probability for both of the buckets that had three hypotheses in them than for the bucket that had only one hypothesis, even though the results seem more consistent with a true answer of .4 than with .5."

"That's correct!  If you spin a fair three-sided coin and then, depending on the results, start flipping a new biased coin with 0.1 or 0.2 or 0.4 propensity to Queen, your chances of getting Text Queen Queen Text Text are around 2%.  Whereas if you just start spinning a fair two-sided coin, your chance of getting Text Queen Queen Text Text is around 3%."

"The 0.4 sub-hypothesis is most likely to generate Text Queen Queen Text Text.  But that hypothesis starts out with only one-third of the prior probability mass in the 'less than 50%' bucket of hypotheses.  The bucket as a whole is less likely to generate the sequence than the sub-hypothesis inside it."

"Incidentally, one would not in Baseline say that the sub-hypothesis of 0.4 propensity started out with a 1/3 'likelihood' of being true, but that it started out with a 1/3 'prior-probability' of being true.  Some of the reason why I've been using those Baseline words is because they have precise meanings, whereas Taldane has a bunch of words like chance and probability and likelihood that all apparently mean the same thing why does this crazy language even have that don't actually answer me unless it's important somehow."

- Wait, Keltham thinks he's right??? But he's wrong!

...Alexandre thinks he's just going to stick with 'the odds of him failing the most important class in his life have gone down', and bedamned to the truth.

She's so confused???

...she's going to raise her hand. Because she feels like she's being confused in a smart way and not a dumb way, for once, but if they've belabored this point for too long then he can ignore her. That's how raising hands works, right?

Points finger at Korva.

"I think I must be confused about how the buckets work. Because, like - say we have one hypothesis that the true value is exactly 50%, and one hypothesis that the true value is less than 50%, and one hypothesis that the true value is more than 50%. It seems like the likelihood - or probability, sorry, I didn't catch the distinction - it seems like it must be more likely that the true value is below 50% than that it's exactly 50%, even though the below-50% space also includes values like 0%, which has obviously already been outright disproven. So - I guess I feel like if .4 is more likely to be right than .5, I don't see why the hypothesis that covers a bigger space, that also includes the value that the true value is most likely to be closest to, becomes less likely just because that hypothesis also includes values that are much less likely to be right than some other narrower hypothesis. - I'm not sure I said all that right."

"Okay, wow, people's intuitions about probability work really differently when they haven't been raised as dath ilani children.  I think probably you just need to invent a whole lot of problems and play around with them, the way anyone - well, the way any dath ilani kids would do as kids?"

"Let's say I spin a fair twosided meta-meta-coin... let all coins be assumed two-sided and assumed fair unless explicitly stated otherwise."

"Anyways, new procedure.  First, I spin a meta-meta-coin.  If the meta-meta-coin comes up Text, I spin an 'objectlevel'-coin five times.  If the meta-meta-coin comes up Queen, I spin a three-sided meta-coin.  Then depending on the result of that coin, I fivetimes spin a biased 'objectlevel'-coin with 0.1 or 0.2 or 0.4 propensity to produce Queens."

Keltham will attempt to whiteboard this:

meta-meta-coin / \ 1/2 1/2 / \ meta-coin (0.5)-propensity / | \ objectlevel-coin 1/3 1/3 1/3 / | \ (0.1) (0.2) (0.4)

"It's actually just true that if you don't know any of the meta-spins, and just see my unknown objectlevel-coin producing Text Queen Queen Text Text, there's roughly two chances in five that my meta-meta-coin came up Queen and picked a biased coin, and three chances in five that my meta-meta-coin came up Text and picked a fair coin."

"Why?  Because when you end up with a biased coin, it's sometimes biased 0.4, but two-thirds of the time biased 0.1 or 0.2.  Mostly you'll see fewer Queens, when the meta-meta-coin comes up Queen.  When you see Text Queen Queen Text Text, that could be because the meta-coin was flipped and selected the 0.4 coin, but more likely, the meta-meta-coin selected a fair objectlevel-coin and that fair objectlevel-coin happened to produce two Queens and three Texts."

"A dath ilani kid would now be 'programming' a 'computer' to run a million simulations of this procedure and show them how many cases of Text Queen Queen Text Text were generated by the meta-meta-coin coming up Queens versus Text and verifying that's how it actually played out.  Here... we'd need to find a three-sided coin and a ten-sided coin, or maybe a cube and an 'icosahedron', regular 20-sided solid, to spin.  And then we'd probably have to do a few hundred spins to collect enough 'data' for the 'statistics', if the math didn't feel intuitive."

"But I'd hope that - the first time the meta-meta-coin came up Queen, and you spun a meta-coin and it selected 0.1, and the resulting objectlevel-coin produced Text Text Text Text Text, it might become more intuitive why, when the meta-meta-coin comes up Queen, what you mostly expect to see is mostly Text.  So when you don't see that, it's not likely the meta-meta-coin came up Queen."

"That... makes sense, but how do we know that there isn't instead a four-sided meta-coin that picks among all the possible coins, including the fair one?"

"In terms of this particular problem?  Because I told you so."

"Why did I tell you so?  Because I was trying to pump the intuition that - assuming 0.1, 0.2, 0.4 equally prior-probable within the 'less than 0.5' bucket - that bucket was then less 'likely' to yield Text Queen Queen Text Text than the 0.5 bucket, even though 0.4 was in that bucket.  I was trying to pump that intuition by showing that, if the whole biased bucket and the whole unbiased bucket started out equally probable, then, after seeing Text Queen Queen Text Text, we'd think the unbiased bucket had become more probable and the biased bucket less probable.  So what we saw must have a lower 'likelihood' in the biased bucket that starts out with 0.1, 0.2, and 0.4 having equal prior-probabilities."

"I mean, we could argue about how that would go in real life, instead of a thought experiment.  You could say that all four hypotheses are equally simple to describe out loud and should therefore be around equally probable.  I could then counterargue that if we're talking about an actual coin, then in real life, most coins are probably pretty close to being fair random-number generators when spun - though I ought to actually verify that here, before I bet anything important on it.  So it should actually take hundreds of observations before we start believing the coin is 40% biased towards Queen, I would argue; five coinflips is nowhere near enough.  Therefore, I'd conclude, 'the coin is biased 40% Queen' is a lot less likely than 'the coin is an unbiased random generator'."

"But it would be better if arguments like that didn't have to appear in our 'published-experimental-reports'.  Which is one angle towards 'grokking' an underlying central reason why 'published-experimental-reports' ought to summarize likelihoods for hypotheses that are more like 'observational likelihood if this coin has 0.4 Queen propensity', and less like 'observational likelihood if this coin has a less than 50% Queen propensity'."

"If you just summarize for the reader 'What is the likelihood of my data, in the world where the coin comes up 40% Queen?  The world of 50% Queen?  The world of 10% Queen?' then you don't have to confront the question of whether 40% Queen was 1/3 as prior-probable or equally prior-probable with 50% Queen."

"Oh, and, uh, to make it explicit:"

Examples of Baseline terms for 'prior', 'likelihood', 'posterior':

'Prior' coin is 40%-Queens:

P( Q=0.4 )    = 1/6

'Prior' coin is 50%-Queens:

P( Q=0.5 )    = 1/2

'Likelihood' of TQQTT, if coin is 40%-Queens:

P( TQQTT ◁ Q=0.4 )     = 0.03456

'Likelihood' of TQQTT, if coin is 50%-Queen (fair):

P( TQQTT ◁ Q=0.5 )     = 0.03125

'Posterior' coin is 40%-Queens, after seeing TQQTT:

P( Q=0.4 ◁ TQQTT )     = 3456 / (729 + 2048 + 3456 + 3125*3)    >(1/5), <(1/4)

'Prior' coin is <50%-Queens:

P( Q<0.5 )    = P( Q=0.1 ) + P( Q=0.2 ) + P( Q=0.4 )   = 1/6 + 1/6 + 1/6   = 1/2

'Likelihood' of TQQTT if <50% Q:

P( TQQTT ◁ Q<0.5 )     = 1/3 * .00729  +  1/3 * .02048  +  1/3 * .03456   ~ 0.02

"How am I doing, Korva Tallandria?"

"I - think I get that, as far as it goes, with the coins? But I don't immediately see how it applies to the people. - if it applies to the people the same way, I'm assuming it does but I'm not sure I should be."

"I mean, the sense in which it also applies to the people, is that your report should summarize the 'likelihood' that your results were generated by a 10% propensity for all-10s to guess within 30 minutes, not the 'likelihood' that your results were generated by a less than 50% propensity for all-10s to guess within 30 minutes.  Because to do the latter thing you have to make a bunch of weird assumptions, and your math is just going to get more and more needlessly complicated as you dig yourself in further."

"And by way of showing how much further into complicated trouble you'd end up digging yourself:"

"Again, let's say we were going by bucketed hypotheses.  One hypothesis, the meta-coin hypothesis, says that there's a 1/3 chance we live in a world where all-10s have a 10% propensity to solve 2-4-6 in 30 minutes, 1/3 chance it's 20% propensity, 1/3 40%.  The other hypothesis, the fair-coin hypothesis, says we live in a world where all-10s have a 50% propensity to solve in 30."

"We don't actually need to consider the probability of these two hypotheses relative to each other.  Let's say we test five all-10 subjects and get NO YES YES NO NO, meaning two subjects guessed within the time limit, three didn't.  Our experimental report is just going to summarize the 0.2 'likelihood' of the data assuming the meta-coin hypothesis bucket, and the 0.3 'likelihood' of the data assuming the fair-coin hypothesis.  That's true regardless of the 'relative prior-odds' of the two hypotheses relative to each other."

"So we publish our report.  0.2 likelihood for the less-than-50% bucket, 0.3 likelihood for the 50%-propensity hypothesis.  There's a questionable assumption that 10%, 20%, and 40% were all 1/3 likely assuming the propensity was under 50%, but fine, whatever, we've got to assume some 'prior distribution' to report a combined likelihood on that whole bucket all at once, yo."

"Along come some replicators.  They test 5 more people.  They get YES NO NO NO YES, so also two subjects who guessed and three who didn't."

"Now what?  What does the combined evidence say?  Anybody want to give the obvious wrong answer?"

"Obvious wrong answer:  This new data also has 0.2 likelihood on the less-than-50% bucket, and 0.3 likelihood on the 50% bucket, so the combined likelihood across the two experiments is 0.2 times 0.2 equals 0.4... equals 0.04 assuming less-than-50%, and 0.3 * 0.3 = 0.09 assuming 50%."

"And why isn't that just totally right?  Or maybe I'm trolling you by calling it the wrong answer, and it is right?  Candidates only, you can message Ione if you're worried your reply is stupid."

"Are we still not considering any other - hypotheses, and just interested in the relative ratios?"

"Yup!"

"We've become more confident, which sounds right... does this get the same result as if we did one experiment with ten people in the first place? I'm not sure if it does..." She's going to start checking the math.

"Well, everyone else do feel free to follow along and try the math on that part.  Raise your hand when done, practice accuracy before you practice speed."

"After all, a Lawful way of looking at the world shouldn't care whether you call your collected 'data' by the name of one experiment or two experiments.  You should always get exactly the same answer either way.  Though, to be sure, you might have occasional occasion to notice that different 'data-subsets' might have been collected under possibly different conditions."

Pilar raises her hand before anyone else.  She's faster than even Asmodia once she knows exactly which rules to follow and which procedure to execute, and in this case she knows exactly what to do.

The first of the new researchers to finish speaks up immediately - "About Fifty-five hundred and sixty-seven to ninety-seven hundred sixty-six," she says, "which really isn't the same thing as four to nine."

"Um, sorry for not being explicit - if I say for people to raise they're hand when they're done, that's so I know everyone is done, and meanwhile, everybody gets a chance to try on their own before hearing anybody else's result."

"At least you didn't say how you did the calculation, so others can, perhaps, do their own calculations and see if they think yours is correct."

Keltham comes over to look at the scratch paper, if any; what calculation seems to have been done?

Squared both sides of 729/100000, 2048/100000, and 3456/100000, divided it by three, then compared it to 3125/100000 with both sides squared, then canceled out the denominator since it was the same on both sides of the equation, with some approximations for doing faster arithmetic.

Message to her:  Don't smile or anything, but:  Correct.  Though two significant digits would've been fine there.

...and she was first of the newcomers, so probably everyone is trying to compute needlessly many digits, which, okay, fine.

Keltham will wait until everyone's raised their hand, and then go to the whiteboard and show how he'd have approximated it:

0.   1/3 * 0.00729^2  +  1/3 * 0.02048^2  +  1/3 * 0.03456 ^ 2   vs.   0.03125^2
1.   ~ (2^2 + 3.5^2)/3   vs.   3.1^2
2.   ~ (4 + 12.25) / 3   vs.   9.61
3.   ~ 5.4 vs. 9.6   (/6 * 10 => proportional to 9 vs. 16)
4.   ~ .00054 vs. .00096

"...where step 1 is dropping the first term that's obviously going to end up insignificant, multiplying both sides of everything by 100, and rounding to two significant digits.  I mean, we wouldn't do that in Civilization because we have 'computers' to do the calculation for us, and here you might not do it in a Science! report, but it's definitely fine in my classes."

"And step 4 is dividing again by 10,000 to put that factor back in, because your experimental report should summarize the absolute likelihood of the data, not just the relative likelihood of the data.  Somebody who wants to compare some completely different hypothesis's likelihood, one you didn't even consider, needs the absolute likelihoods to do that - they need the .00054 and .00096 version, not the 9 vs. 16 version."

"Imagine that we've got two separate research groups both testing the hypothesis that all-10s solve 2-4-6 with 50% propensity, or alternatively, less than 50% propensity.  They each don't know the other group exists; however, they both use the - hypothetically for this thought experiment - universally standard rule that 'less than 50% propensity' is of course best-modeled in practice by three equally weighted 'probability point-masses' on 0.1, 0.2, and 0.4."

"The first group reports a likelihood of 0.02 on the less-than-50% metahypothesis, and a likelihood of 0.031 on the 50% hypothesis."

"The second group reports the same thing."

"The way we've set up the hypotheses being reported on, we cannot just multiply the two likelihoods together.  The task of combining evidence from different 'published-experimental-reports' is now a big complicated deal requiring us to recheck their original data and redo all their calculations."

"Alternatively, if they had both reported likelihoods of 0.007, 0.02, 0.035, and 0.031, on the distinct hypotheses of 10%, 20%, 40%, and 50% propensity respectively, we could have just multiplied the likelihoods together from both groups, and our ability to accumulate data from across multiple experiments would be vastly simplified."

"Of which it is said out of dath ilan, to those dath ilani children who need to hear it:  Different 'effect-sizes' are different hypotheses."

"That, Carissa, Pilar, is why we can't just have the hypothesis that all-14s have at least five times the propensity of all-10s to solve 2-4-6 in 30 minutes.  We can look at the data and see if that actually happened or not.  But as soon as we try to figure out the exact likelihood that it happened, we are cast into a nightmarish multiverse of different ways the world could be, such that the statement 'all-14s are more than five times as likely to solve in thirty as all-10s' is true about worlds like that, all of which have different likelihoods of yielding the data we saw."

"Like, just on this breakdown, that could be because the chances were .2 and 1.0, or .1 and .5, or .1 and .6, or .1 and .8.  And every one of those hypothetical propensity '2-tuples' will yield a different likelihood for whatever data we saw.  So you'd have to put a prior on their relative odds inside that metahypothesis bucket, before you could calculate the likelihood for the whole bucket."

"And then, actually seeing any data, would update the odds inside that bucket, which would change the likelihood for any future experiments, even if the replicators saw exactly the same data you did."

"And that takes us to the principle I wrote earlier on my todo list:"

#2 - Separate experiments are usually supposed to avert 'conditional-dependencies', watch out for when that isn't true

"What I mean here is that - when you are otherwise doing things correctly - it should usually be the case that, for the likelihoods that the 'published-experimental-report' is summarizing for different hypotheses, if some replicators came along and did their own experiment, their likelihoods should be something they can calculate independently of your data.  It shouldn't be the case that they have to look at your data, to figure out the likelihoods given their hypotheses."

"This in fact is the property that lets us compute the joint likelihood of a hypothesis across two experiments, by multiplying the likelihoods together from the separate experiments.  Symbolically:"

P( data from first and second experiments ◁ the hypothesis )
= P( data from 1st experiment ◁ the hypothesis ) * P( data from 2nd experiment ◁ the hypothesis)
    if and only if
P ( data from 2nd experiment ◁ the hypothesis ) = P( data from 2nd experiment ◁ the hypothesis & data from 1st experiment)

"When you say, 'maybe all-14s have 60% propensity to solve in time, and all-10s have 10% propensity to solve in time', you're describing a way reality can be, where the likelihood of my found pattern of YES and NO responses, if that's true, is just the same no matter what data you found in your own experiment.  Maybe your data made that world look incredibly improbable, but that doesn't matter; I can still answer the question of how likely my data would be, if that world was the case, without looking at your data."

"When you say, 'maybe all-14s have at least five times the propensity of all-10s to solve 2-4-6 in 30 minutes', that is a way the world can be; but it's a way the world can be, where calculating the likelihood of my data in that world, requires me to make up a bunch of prior-probabilities, and then those probabilities change depending on the data that you got."

"Which makes it immensely complicated to quickly look over the summaries of what different people's experiments said about different worlds, and combine them together into a joint summary of what reality has told us about them all."

"It would, in fact, be possible to combine a lot of little experiments all of which suggested that - if you wrote the summary this way - the data was more likely if all-10s had 50% propensity to solve, versus less-than-50% propensity to solve, and get out a new update that the combined data was more likely if all-10s had less-than-50% to solve.  If you multiplied enough 0.035 likelihoods from the 40%-propensity hypothesis, compared to the 0.031 likelihoods from the 50%-propensity hypothesis, then eventually the 40%-propensity hypothesis would come to dominate the predictions of its bucket, and then its bucket would start to dominate the other hypothesis."

"Which paradoxical-seeming combination, again, doesn't happen if you consider the 40%-propensity hypothesis separately, because then it's clear from the start that 40% propensity is gaining on 50% propensity in each experiment."

"Hence again the proverb:  Different effect sizes are different hypotheses.  Which argues again against thinking that 'all-14s are at least five times as likely to solve as all-10s' is a good way to split up the world for purposes of SCIENCE!  Even though, in terms of 'truth-functional' scaffolds, it is a way the world can be.  It could even be the metafact that is useful and that we're interested in.  We should still ask the Science! question first, what are the exact real effectsizes, and then check the useful metafact afterwards."

"Likewise if you started thinking that 'this coin isn't random' or 'this coin is biased to favor Queen' was a good way to describe the hypothesis you were considering.  If two experiments show that the same coin is probably biased to Queen by notably different amounts, they're pointing at incompatible ways the world can be, and something is wrong, some condition has changed between experiments, at least one group is screwing up."

"You definitely wouldn't say, 'Well, our hypothesis was that this coin was biased to favor Queen, and group one spun it a bunch of times and found that it came up Queen 900 times out of 1000, and group two spun it a bunch of times and it came up Queen 520 times out of 1000, and both of those results are instances of 'the coin came up Queen more often than it came up Text', so both confirm the hypothesis that 'the coin is biased Queen', and the experiment has 'reproduced'.  You are actually less confident after two apparent confirmations of your original statement than you were after one confirmation, because in the details of the particular worlds, it's clear that something was wrong with at least one experimental setup."

"But that apparent paradox is just an artifact of bucketing together different ways the world could be, that yield very different likelihoods on the exact data observed, into one metahypothesis of 'this coin is biased to favor Queen'.  If you said instead 'the coin yields 90% Queens' or 'the coin yields 52% Queens', there would be no illusion of the first experimental result agreeing with the second result, there would be no illusion that the result had 'reproduced'.  Fix a local hypothesis, a single effect-size in this case, that makes the data have independent likelihoods between one experiment and another, that fully specifies the likelihood of the data as a matter of logic, and doesn't change when we read other experimental reports.  Summarize the likelihoods for hypotheses like that, and it would be clear that the data from one experiment was compatible with an exact hypothesis, and the data from the other experiment was not."

"Which, uh, yeah, the lesson is, there are these careful precise details about how to do SCIENCE! correctly, and those details actually matter a lot for making your whole Civilization's SCIENCE! output fit together and have the whole thing make any sense.  Even for a small project like ours, it's still probably best to do it that way, if we want things to make any sense."

Korva, who was the last to raise her hand for the last exercise, and who has been calculating through another panic attack for the past twenty minutes, has now realized that the horrible feeling of wasting lots of time making errors that she still doesn't understand on stupidly complicated problems was the point of all of this, that this incredibly painful classroom experience has all been an illustration specifically for the benefit of Carissa Sevar about why they shouldn't do things the way she suggested, even though there is no outward indication that the Chosen of Asmodeus herself experienced any distress about it at all at any point.

Korva thinks that, probably, if she were in charge, she would have experimenters report their data, and then everyone who wanted to see how well that data fit some specific hypothesis could CALCULATE IT THEIR OWN GODSDAMNED FUCKING SELVES.

There are, even in dath ilan, children who will agreeably acquiesce in doing SCIENCE! the same way everybody else does it, without carefully detailed demonstrations of exactly what grimdark dooms will befall them if they try to make up their own methodologies that violate the coherence constraints.

Tiny Keltham was not one of those children.

And besides, even if you report the raw data like a sane person, people do need to know which calculations to do after they've got all the data, right?  There are some local calculations that do tesselate together neatly to global calculations, and you might as well summarize those when they exist.  Which requires you to know how to set up the kind of calculation that modularizes well, and distinguish it from calculations that don't.

Willa feels like she's following well, but she did think she was before, and then she still managed to get herself tied up somehow. In front of everyone, as usual.

Though in retrospect she thinks she had it but there was just a lot of confusion about before-chances priors and some miscommunication? But overall she's happy that this particular weird mess of logic is going to stay away from SCIENCE! The bad buckets were hurting it.

She doesn't know SCIENCE(!) well enough to feel protective of it yet, but she expects she will. Except that expecting yourself to be convinced of something in the future doesn't make any sense. So she might as well feel protective of it already.

Alexandre is apparently quite good at probability theory, since he was following for all of that! He is therefore happy that his knowledge (a) is growing and (b) will probably continue to grow, since he's not permanently behind everyone and can therefore continue taking the MOST IMPORTANT CLASS IN THE WORLD so he can obtain MASTERY OF REALITY. Alexandre's power and knowledge continue to grow, all hail Lord Asmodeus.

"Also, just to be clear, while you wouldn't necessarily get paid very much for it except in very special cases, there is such a thing as just running out and trying a bunch of stuff that you didn't preregister and aren't going to be especially careful about analyzing if you even report the results at all.  That's definitely still 'science', the individual kind, even if it's not 'SCIENCE!' the project of all Civilization.  It is not considered foolish to do noncareful nonrigorous experiments to figure out which careful rigorous experiments to do, before you spend a lot of effort on writing up something to preregister."

"That's, uh, basically what I did the previous day when I was experimenting on boiling acid and having it explode a lot.  I wasn't exactly trying to lay a firm foundation for future Golarion Science!, just poking around rapidly and trying to get a handle on how 'chemistry' worked with magic in the mix."

"This, too, is something of a dath ilani stereotype.  Stereotypically, it sometimes involves an Ione-like person who tries to prevent the maniacal-scientist from accidentally killing himself.  They are even, often, a girl and a boy respectively, and it is not unknown for them to end up romantically involved.  Though I have to ask again, do I really need one of those when I already have several girlfriends and my new universe comes neatly equipped with healing and resurrection spells?"

"YES because resurrections are EXPENSIVE and they are MUCH MORE EXPENSIVE when the person has COMPLETELY DISSOLVED THEMSELVES IN ACID."

"Oh.  Huh.  I wasn't actually aware that it got more expensive if I completely dissolved myself.  Good thing to watch out for, then."

Why is her life like this.

Even Nethys can't have been this bad.  He wouldn't have survived to become a god.

...there may possibly have been several romantic interests involved in his continued mortal existence.

"I"m actually a little surprised there's not more specialization. If I were imagining how to do this I'd have some people write detailed experiment-specifications based on what they wanted to learn, other people follow the experiment-specifications, and then the first people or other specialists do analysis of the results to figure out how to bet."

"It's specialized out to the Fourthplanet colony and back again.  There's going to be literally hundreds of people in Civilization who do nothing but talk to the big chemistry outfits to figure out exactly what they need and write up exact experimental specs for experiments involving 'sulfuric acid' in particular."

"Thing is, everybody in that process needs to understand the rules on how the greater Science! process works and why.  You can't write up a good experimental spec unless you know how a replicator will carry it out and what an analyst will do with the data.  You can't be a good analyst unless you know what was really going on in the experiment, what the impact-buyers were originally asking for, and what the report-writers need from you."

"Good specialists don't have to be good generalists, they can be mediocre generalists, but they do have to be generalists.  You don't need to be able to write a pro-level saleable experimental-report before you can hire a writer to do that for you.  You do need to be able to write a terrible, unreadable report that is nonetheless judged by a professional skill-evaluator to contain all of the necessary details."

"There's literally nobody in the process who doesn't need to understand the Law of Probability well enough to know the difference between hypotheses-that-fully-imply-likelihoods and semantically-well-formed-propositions-that-underspecify-likelihoods and word-sequences-that-don't-even-have-clear-truth-functionals-yet."

"Remember, I am not in fact, by Civilization's standards, a great experimentalist, a famous analyst, the one who brings the light of cause-selection to the world."

"I'm a teenager what got tossed into another universe, is what I am."

"Everything I'm teaching you is what Civilization makes sure to teach all of the eight-year-olds, so they'll be able to read the newspapers when they grow up."

"I mean, not literally.  Not literally actually.  Not every eight-year-old has read a couple of novels in which the protagonist has to improvise scalable production of sulfuric acid, in which, unfortunately, most of the memorable drama was about safety precautions obviated for us by Resist Energy (Acid).  I know more than the bare basics, in a few places, because I thought they seemed cool to know."

"But definitely everything we've been covering today has been like that."

"There's a saying out of dath ilan, 'Professional specialization is what grownups do when they actually want results.'  But don't go setting your eyes on a specialist's glory just yet.  Let's all get to the eight-year-old generalist level first."

"If, like... we can successfully do even that, without me having actually been a professional specialist in teaching eight-year-old Law of Probability."