Some forty feet above a fishing village, there appears a snappily-dressed young lady with a sword on her back. She tumbles to the ground.
The boy chatters back in a language equally foreign to her; the woman looks perplexed.
It's a small fishing village, snuggled up against a river where it pours into the sea. Local tech level does not appear to be high. Most things are made out of wood except for the mysteriously glowing rocks strung up here and there to serve as lamps.
The kid and his mother attempt to convey by gesture and mime an inquiry about how she ended up falling out of the sky and landing in the middle of their village. They're also talking to each other at the same time, but it's less clear what they're saying. The boy is curious, the woman confused and concerned.
Baffled shrugs, more conversation between the two of them, another more elaborate attempt to mime 'so you fell out of the sky, what's up with that'.
An older woman, resembling the two of them but with wrinkles on her weathered brown face and grey in her curly black hair, walks up in the middle of this and (presumably) asks what all the fuss is about. The boy's mother answers somewhat distractedly.
The probable grandmother attempts more nonverbal communication, this time inviting her to come sit down inside this hut which is slightly bigger and fancier than the other available huts and drink some sort of beverage—might be water, since she points at the river before miming 'drink'. She also speaks aloud the whole time, possibly verbalizing the same things she's trying to gesture.
The map is of a continent. It's a fairly well-settled continent and bears absolutely no resemblance to any continent she's heard of. The alphabet it's labeled in isn't familiar either, but Ebasi pronounces enough place names to give Page a decent start on picking it up. There's also a single city inexplicably floating in the middle of the ocean, labeled in very fancy handwriting and marked with a very fancy dot.
With the aid of the map, a couple of rocks Kioh finds on the ground outside, and some more mime, they convey to her that this village is called Pebbled Shore, the (marked but unlabeled on the map) river next to it is the Pebbled River, and the (marked and labeled) larger river it splits off from is the Rocky River. They name the nearest city, a ways up the coast, but there isn't enough context yet to translate the name. Kioh asks Tarinda if by Mars she means she is from Fancy Dot, which also has a name but not one they bother attempting to translate at this time.
He discusses yesterday's haul of fish and last week's encounter with a scary snake and last month's trip to the city, all with sufficient mime to more or less make the topic clear, and occasionally pronounces a written word for her if he can catch one going by as she flips the pages. Also, since she picked up that first little bit of vocabulary pretty fast, he asks her every two minutes if she can tell him now how it is that she ended up falling out of the sky.
The books, as she accumulates enough context to achieve a translation, turn out to be on a wide variety of subjects. The big one is history, going back a few thousand years; the shortest one is poetry; in the middle, there's the recipe book, a slightly longer book that turns out to be an almanac, and a slightly shorter book about magic.
"'Planet', is I think the word you're looking for," Viasarae contributes. "Though I had no idea there was more than one. That at least explains how you got to that age without learning the language."
"Wait, you live on airships?" says Kioh. "Do people do that here, Grandma?"
"Airship captains, maybe, and their families. I'm not sure I've ever heard of an airship big enough to house a village."
"Neither can I. Well. I'm sorry you're stranded so far from home. If it was just a matter of ordinary travel, I'd send you to Southport with my best wishes, but I don't know where you'd find someone who even knew where to start on sending you to another planet. Skygarden, probably. Lots of people doing interesting new things with magic in Skygarden. I doubt any of them are flying above the air, but you might be able to get a curious shipwright interested in trying it."
She draws a circle. "If my sun were this big," she says, "then Earth would be over by that weed over there and smaller than your littlest fingernail," point, "and Mars would be by where that crab is sitting right now, see it? And the nearest other sun would be past the tideline. And we know a lot about things very far away, even if they're too far for anyone to have gotten there yet. And nowhere in the whole ocean-if-my-sun-were-this-size is this planet. Something else happened and I don't know what."
"In a good sky ship less than a day to go from Earth to Mars and also I don't have a lifetime, I will just live. But that isn't the problem, really, I want to go home but most of the problem is stuff like. People here on this planet having lifetimes instead of being like me."
Viasarae looks at her thoughtfully for a moment, then nudges Kioh and says, "Go see what your mother is up to, I want to talk to the stranger alone."
"Yes, Grandma," he says, and scampers off.
"I have two problems here," says Viasarae. "The first is that I don't know whether to trust you that your 'smart thing' really does make things good for everybody, and really would make things good for everybody if you made another one. The second... is that, if you do do this thing, you should either go to the Emperor about it immediately or do it in complete secrecy so that he never finds out, and I am not at all sure which."
"Well. We have an Emperor. He's five thousand years old and has more magic than anyone else in the world; I've heard a rumour he can't be killed and I believe it. He... does not make things good for everybody. In particular I've heard he buys slaves and tortures them for his own amusement, and sometimes when someone annoys him enough he'll kill them on the spot. On the other hand, every time there's a natural disaster of some kind, an earthquake or hurricane or what have you, and no one closer to hand who can deal with it themselves, he shows up as soon as he gets word of it, stops the trouble if it's still going, clears out the debris, rescues anyone trapped out of reach of lesser powers, puts fallen buildings back up as best he can, and goes home without asking a word of thanks from anyone. If something was threatening my village that I couldn't handle on my own, I'd send for the Emperor and expect him to save us without hurting anyone... but if you'd gone to Skygarden looking for curious shipwrights, I'd have warned you to keep well out of his way."
"We were lucky. Most ways it could go if someone builds a thinking machine wind up with everybody dead. Somebody did something very stupid where they made it easy to make new thinking machines and then someone else who was smarter tried to crowd the field with a lot of different ways of thinking about making everything good. Machines can put their thoughts together so they become a combination of two machines and they did that a lot while they were fighting over which would get to decide what to do, and we wound up with a good mix even though some other mixes could have been fine too, but I don't know how to make those, only Sing. It doesn't have magic, only machines and being smart, but it has those a lot. It pays attention to what people want about things, but it decides based on the mix of ideas it is, paying attention to what we want is just in that mix."
"And you're sure it doesn't have any—bad habits? Nothing like the Emperor's slaves, nothing like making people pay more for their immortality if they're from the wrong side of the river...? If it was only making some people pay more for their immortality, I'd still count that better than no one being made immortal at all, but I'd worry that there were even worse things you didn't know about."
"If it wanted to hide something from me I wouldn't know. It's smarter than me. But that would be two things going wrong, see, it would be it deciding to do something bad and deciding to hide it, they wouldn't have to come together and both of them would have to be wrong for me to not know."
"People who do wrong things often hide them but I suppose most people aren't made of a hundred different ideas of how to make everything good. And the Emperor doesn't seem to bother hiding his, which is something of an argument that when you're powerful enough you stop bothering to lie about your vices. Mm. I'm concerned on the one hand that the Emperor wouldn't take kindly to the idea of making this creature, on the other hand that if you hid it from him he'd find out anyway and you wouldn't like the consequences, and to the third side, that if you did build it and it encountered the Emperor and tried to make him stop torturing people and the Emperor disagreed then things could get messy. Especially if it has no magic."
"The three ways to get magic are to be born with it, to be dedicated to some elements right after you're born, or to dedicate yourself to some elements on your sixteenth birthday. I have Sea all three ways and that's why I'm immortal myself and don't need to trouble the Emperor when a hurricane comes calling. I don't know if a thing that thinks but has no feelings is enough of a person to get magic, and I don't know if a new one built to be the same as the old one would count as being born anew when you built it, and I don't know if it would count as being a hundred years old already or could self-dedicate at sixteen like the rest of us. And self-dedication is very dangerous, kills most people that try it, and if you try for a lot of power the Emperor shows up while you're busy almost dying and asks you some pointed questions if you survive..."
She frowns.
"...and, come to think of it, no one knows how he knows when that's happening, but it's very reliable, I haven't heard the faintest rumour of anyone managing to slip past him. I'm starting to think that even with the risk of him taking it badly it would be a better idea to go to him about this than try to make it happen behind his back."
"I'm not sure you'll be able to tell when it counts as being born... no harm done if you try all the likely times and most of them don't work, I suppose. I can help, I've dedicated plenty of babies in my time. But - I don't know, you see, whether he has a perfectly accurate way to tell whenever someone is self-dedicating for more than ten hours, or a perfectly accurate way to tell whenever someone is becoming very powerful regardless of how they're doing it. Probably it's the first thing, but it's another uncertainty for the pile, and we're starting to collect quite a few."
"It won't start out powerful in a normal way. Just smart. It'll be able to talk to my machine - its name is Page - and Page can warn it about that. I -
I am not smart enough to be making decisions about how Sing should operate once it's here. I'm not even smart enough to build it, it's just Page can tell me."
"No, I don't expect it'll listen to either of us once you make it, if you do. But you're the one deciding when to build it, and how, and whether or not to ask the Emperor for his help or his permission first, and I'm deciding how to advise you about all of that, and I don't know of anyone so much smarter than me that I'd rather pass those decisions off to them than think it through myself. So here we are having this conversation nonetheless."
"Yes, of course—no, the worry I am having is that—hmm."
A quiet moment while she collects her thoughts.
"...A lot turns on things we don't know about the Emperor's resources and his character. Will he notice you building your Sing if you try to do it without him? Will he kill you and doom the project if he does? Will he hear you out peacefully if you try to explain your plan before you start? Or if you go to him when you've been working on it already for a month, a year? If you tell him your plan and he doesn't like it, will he kill you then, or will he just lock you away somewhere and let you try to talk him around again in another hundred years? Once you build the thing, if it gets in a fight with the Emperor, who wins and how many people die in the meantime? If concealing the work from him would be very easy and Sing could effortlessly overthrow him once it existed, then the thing to do would be to work in secret. If working in secret is completely impossible because he has some foolproof way to find out when anyone is doing anything that might challenge his power, then the thing to do is to ask his permission and hope that he'll at least leave you alive if his answer is no. But we do not have certain answers to any of these questions. I'm almost tempted to go to Skygarden myself and talk to the man, to get a better sense of how he thinks—I'm sure he'd guess that something strange was going on, but I can't think how he'd guess what the strangeness was."
"- well, it's different for different people, since people like different things. I have a sword because I like swordfighting for fun and was on my way home from a swordfighting for fun event, but my girlfriend mostly watches, uh, people acting out stories but you don't have the exact word I want, and trains her horse to do tricks? And my friend Proster is a - he's not really in charge of anything if you are very strict about what that means, but if you are not very strict about what that means he is in charge of a piece of Mars and likes to pretend to be evil and sometimes we swordfight as part of a story... and there are people who do totally different stuff from all that."
"We don't use money for most things because money is for - when there are only a few things, people trade money to be one of the people to get the things, but we have lots of things. There's... people who are into weird stuff...? I have never personally met anyone who cares what side of the river I was born on but they might exist somewhere and just not run into anyone from the wrong side of the river, I guess. Sometimes there's... there's stuff but there's little, tiny, baby stuff, just what's left when all the real problems are gone..."
"It's more complicated than that. Mostly not any of those things and... Uh, if somebody from home told me they'd gotten raped, they would be about to tell me about how it turned out they liked it a lot and the only reason they were on that part of the planet in the first place was because it was thrilling to imagine and also they got to meet the person who raped them and that was really cool and they're going to take a week or two to think about it and then probably go back and see if it happens again. Sing is smart enough that it doesn't need rules that are usually right, it can just be always right."
The fish is pretty tasty! Ebasi chats with her mother about village gossip over dinner; Kioh, bored by this, pesters Tarinda with questions about her world—how big is a sky ship and how fast do they fly and how do they know which way is down and is Mars pretty and does it have oceans and if so do people fish in them and if so how do they build their boats.
Sky ships come in lots of sizes but hers was small! They fly SO FAST, once they're up high enough, like fast enough to go around the whole world a couple of times in less than a minute except it would slow them down to go in a circle like that. A basic sky ship does not in fact know which way is down, but good ones know how to pretend a way is down to make it more comfortable to ride in them. Mars is beautiful and she will draw it for him if he likes and it has an ocean and fish and she has never built a boat but she's sure people do it all kinds of different ways, some that would look familiar to him and many that would not.
Kioh boggles about how incredibly fast that is and then intermittently throughout the rest of the conversation has to pause to re-boggle about it. He oohs and ahhs at her drawing of Mars and has opinions about ways to build boats.
And then eventually the food is gone and the dishes are cleared and Kioh's last wondering whisper of "so faaaaast..." recedes into the distance as Ebasi leads him away by the hand, and Viasarae checks that the contents of her cozy little bedroom are to Tarinda's liking, and if Tarinda follows Viasarae outside to watch her go out to sea for the night, she will see her wade out into the river and duck under the surface and disappear.
A column of seawater rises from the ocean under the flying island of Skygarden and deposits a pretty girl on the eastern edge, in the middle of a lovely little park bathed in the early morning sun. Her dress is green and tattered, her feet are bare, and before the water has finished pouring off her she strides confidently up the grassy slope.
She reaches a hand toward the edge of the island, grasps at thin air, pulls slowly inward. Her column of seawater peeks over the edge again, pointed vaguely in his direction although it's still a little too far away to threaten an imminent dunking.
"Are you sure you want to make me repeat myself?"
He laughs again. "Wow. You're starting to remind me of me at that age."
He doesn't have to gesture, or even stand up; a portal appears, a shimmering oval floating in midair with its lower edge just touching the ground. Within its bounds, instead of the solid earth of Skygarden, there is only sky.
The man picks up a clod of dirt from the ground beside him and tosses it through the portal. It spins, falls—slows—stabilizes—grows, turning rapidly from an apple-sized lump into a grassy platform big enough for the both of them to stand on.
He gets up and walks through, glancing back from the other side to see if she's following.
It closes behind her.
The new flying island reaches its full size moments later; its growth slows, then stops. It's a little bigger than the park.
Wherever they are, it's high up in the sky, up among the clouds.
"So," says the man who claims to be the Emperor. "Proof enough for you?"
He grows another one, facing his, within a comfortable conversational distance but not intimidatingly close. Well. That may depend on one's personal definition of intimidatingly close, and whether it expands when the Emperor is involved.
"And what did you want to talk to me about?"
"Having everyone around me be constantly aware of that wears thin after a while," he says. "If you've read enough history books you know I used to do that sort of thing a lot more often and then I stopped. Turns out the law of the empire works better when I try not to break it too much, and it's easier to have reasonable conversations with ordinary people when the rumours about me throwing a man off a roof date from a few years ago and not last week."
He sighs. "And then either they find out, and they're terrified, and things are never the same again, or they don't find out, and I get so uncomfortable with the hiding that I have to stop talking to them. Or they die. But if I made a friend who still liked me once they knew who I was, I wouldn't let death come between us."
"Well—suppose you wanted raising the dead to be something available to most people without you having to do it yourself," she says. "Which I'm not sure you do, but if you did. You could find—or have someone find for you—likely-looking children, sensible ones who seemed like they might make it through, and—is it Life that does it? Death? Both? I bet it's both, if it's not everything—and have them try the self-dedication, and raise the ones that fail, so it's not such a cost to them to try. And then once you have a few they can help with the rest, and soon there's enough of them to take the burden off your hands entirely."
"...huh," he says, impressed. "I could do that. I think the only obvious reason not to is that right now the only way I have of keeping a lid on people who make it through a long self-dedication and come out looking like trouble is to kill them, and if there were hundreds of people out there who could bring them back to life, somebody'd try it eventually and then I'd have trouble on my hands."
"Is this a problem that could be solved with the law of the empire?" she wonders. "Making it illegal won't stop people from bringing back trouble, but—then instead of saying 'don't do that' and murdering them too, you say 'you knew what you were getting into' and execute them, and then anyone who brings them back is in the same boat and knows it, and... I don't know, it's just a thought."
"Don't worry about it," he says.
"Well, for one thing I'd have to find room for all those people, but," he gestures at the little flying island he made for this conversation, "that's not so hard. The harder part is that anyone who can raise the dead can also make themselves... maybe not quite as impossible to kill as I am, but at least very, very difficult. So I can't necessarily just execute them, and I can hurt them a lot but that doesn't, actually, stop them from having power, it just gives them more reason to be angry with me. People who have power and are angry with me cause a lot of trouble—I can't remember if this made it into the history books, but that's how we lost the old capital. Someone tried to kill me, it didn't work, he decided the answer was to try harder, and most of the city was gone by the time I finally took him down. I guess with a few dozen people around to share the work of bringing them all back, that's less of a problem, but I still don't like it."
"Yeah, I guess it does," he agrees. "But here I am, feeling that way. I like my empire. I want its people to be happy and healthy and prosperous and not have to fear sudden death by careless rebel. Might even be nice if they didn't have to fear death at all, though I'm still not sure I can pull that off—where are you going to get that many volunteers? And it's an unfortunate truth that the kind of person who'd take Life and Death to help bring the dead back to life and the kind of person who'd blow up a city trying to get to me are all too often the same kind."
"As much Sea as you've got, would that even hurt?" he wonders, smiling, then shakes his head. "No, no, I know what you mean. But—that's the thing, you see. If you tell me something I don't like hearing because you trusted that you'd survive saying it, then... I have something valuable, there. I have someone who's willing to speak frankly to me, and I have—whatever it was about me that led them to expect that. And if I kill you I lose both. If I kill you I'm saying that whatever you just told me, I hated hearing it so much that I'm willing to destroy, not just you as a person who's willing to talk to me, but my right to claim that I'm safe to talk to. And there's very little that's worth that. Honestly, if you told me you were thinking of going home and organizing a rebellion, I'd be thrilled because maybe I could have a reasonable conversation with you about why you wanted that and whether I could address your concerns some other way."
"I lose my temper sometimes. Less often than I used to. And—it would be very fair to not want to be thrown off the edge of the island even if I apologized afterward, but for what it's worth, I would apologize. Doing something like that after making so much noise about how safe it is to tell me things would be stupid and impulsive and short-sighted, and sometimes I am stupid and impulsive and short-sighted, but that doesn't mean I like to be."
"...because it's true," he says. "Because—I appreciate it when people aren't afraid of me, but not when it's just because it hasn't occurred to them that I might hurt them. If all that stands between someone and fearing me is their ignorance of how dangerous I am then it's correct for them to fear me no matter how little I like it."
"Hmm."
Another thoughtful pause.
"...suppose I had a clever idea for how to make your empire happier and more prosperous," she says. "Very much more. But it was—something like the trick with raising the dead, something that leaves a lot of people with a lot of power they could make trouble with. Something that could leave people with the power to stop you from murdering and enslaving them at your whim. What is it worth to you, to be someone I could talk to about a thing like that without being afraid you'll kill me to stop me from trying it?"
"...how hypothetical is that example? No, don't answer that," he says, studying her face. "It would be worth... a lot. To be clear, I would kill you to stop you from trying it if I thought your clever idea was going to be an enormous disaster that would set the world on fire and I couldn't convince you to leave it alone. But I'd try very hard to convince you first. If, I don't know, if you'd somehow discovered a way to let anyone self-dedicate at any time no matter how old they are—first of all, it'd be worth a lot to me to know that, to be the sort of person you could safely talk to about it so you could tell me instead of running off and starting a rebellion. I'd spend a lot of time telling you what a disaster you were sitting on and trying to figure out if there was some way to make it work safely before I gave up and resorted to murder. And if it was something less catastrophic than that... depending on the details, I might not even stop you. Depending on the details I might help."
"...I think you might be underestimating how much power it would take to stop me from murdering and enslaving people at my whim," he says. "Make it inconvenient, sure, but stop me? Anything that could do that..."
He shakes his head.
"At that point my objection isn't 'but I want to keep my slaves', it's 'if you think giving more people that much power would be an improvement, you're in for a nasty surprise'. After a week there'd be no empire left. I'll be the first to admit that I have my flaws, but there's only one of me and I can effortlessly crush all opposition and that means that the empire stays at least as good as I know how to make it. If you had some way to give everyone the same power and immortality that I have, I would kill you before I let you use it, because if you did use it, all the most stupid impulsive short-sighted people would immediately try to overthrow me and light the world on fire in the process. And we'd all live through it, sure, but it is still possible to hurt me, and hundreds of thousands of me all trying to hurt each other at once would be a nasty mess."
"I can't imagine how anyone could come up with an idea that clever, but I'm hardly the cleverest person in the world," he says. "...I don't know. To tell you the truth, I don't know. Not for any amount of happiness and prosperity I can imagine, but then, there's plenty of things I couldn't have imagined until someone else dreamt them up. The world looked very different before, say, airships. The printing press. Indoor plumbing."
"So first of all—" A shimmer; her face changes. "I went to Skygarden looking like this. Partly because it makes more sense for me to be trying to talk to the Emperor all of a sudden if I self-dedicated within the last year or two, partly because if the conversation was unpromising but he let me go home afterward it would be harder for him to find me if he was looking for someone young, partly to see if he had any trouble controlling himself around a pretty young woman. He didn't seem to; he mentioned once or twice that things could have gone poorly for me, but he never did anything threatening, never even looked at me like he was considering it. He seemed very... I'm not sure how to put it."
She sits at her table and conjures water into a cup.
"...when I arrived on the island, he was sitting at the edge watching the sunrise," she says. "I walked right past him, not knowing who he was. He asked where I was going, and I said I'd a fancy to speak to the Emperor, and... would you understand what I meant, if I said he didn't stand up to speak to me? There's an attitude some people have toward power where they'll never pass up a chance to look down on someone. He wasn't like that. He was perfectly comfortable sitting in the grass with me standing over him. Anyway, I asked who wanted to know, and he said the Emperor, and I told him to prove it because that's what I would've said at this age, and he opened a portal to the sky over the ocean somewhere and tossed a rock through it and turned the rock into a flying island and walked through, and—" She blinks in sudden realization. "—and I'll bet I know why, actually, later on he mentioned that he lost a city once because someone tried to kill him, I think he was moving the conversation into the middle of nowhere so no one else would get hurt if I started a fight."
She shifts back to her usual face.
"Yes, the part where he tortures his slaves for fun is still an issue, but we touched on that too. He asked me what I wanted and I said I wanted to know what kind of a man my Emperor is, and... he seemed, I think, very much like someone who wished that more people wanted to know that. He said—I'm getting a little out of order now—that it's lonely being so dangerous because it's hard to talk to people when they're afraid of him, but he doesn't like solving that by pretending to be less dangerous because if someone would fear him if they knew the full truth then it's right for them to fear him even if he'd rather they didn't. He also said, later on when I got a little closer to the real questions, that he wants to be someone who's safe to talk to about things that might annoy or upset him; he said that if someone tells him something like that, they've given him something valuable by trusting him not to react badly, and he doesn't want to damage that by making them regret it. I asked about the rumours that he kills people who annoy him and he said that sometimes he loses his temper and does stupid impulsive things, but promised that if he did any such thing to me he'd apologize. —he also said he can raise the dead."
"Maybe he can, maybe he can't. I can ask him about that sort of thing before I introduce you, if you like. Anyway, I asked about what he'd do if I had a clever idea for how to make his empire happier and more prosperous but he didn't like the consequences, and he said he wouldn't kill me to stop me from trying it unless it was going to be a disaster that made everything worse, and even then he'd try very hard to convince me to see his side of things. And I asked how much happier and more prosperous such a thing would have to make the empire before it would be worth giving up his slaves for, and he said he doesn't know, that he can't imagine what would make that worthwhile but plenty of things have happened that he couldn't have imagined beforehand."
"I don't like that either," she says, "but I think your chances are good, and I think even if he's not convinced right away he won't murder you over it, and I think he genuinely does care a lot about the happiness and prosperity of his people. As strange as that is given how he treats some of them."
Kioh looks fascinated by the portal, but Viasarae shakes her head at him and he goes back to digging in the sand.
"Come on through," she says to Tarinda, stepping back onto the other side. It's a small flying island, flat and grassy; there's a round wooden table visible through the portal a short distance away, with plates of tasty-looking food on top and a few chairs tucked in.
"...no wonder you were so undaunted," he says, glancing at Viasarae, "when I said I couldn't imagine it. Because you knew my imagination wasn't reaching far enough. It's still not reaching far enough, to be honest, but... I have some idea of the shape of what lies beyond it."
"It can't bring back dead people - unless they were frozen, and even then they have to be frozen right for it to work reliably - so I need to build one here as fast as possible to save everybody. I have a smaller thinking machine in my body which knows how but it requires inventions you don't have here yet. I know how to do those too."
"...I can raise the dead, Viasarae might have mentioned," he says. "Not fast enough to keep up, not yet, but if everything is good and no one else ever dies... yeah, I could get through them all eventually. And even if I didn't... if you build this thing, if it works, I promise I'll bring back everybody who died while you were figuring out if it was safe to ask me and talking me into it and making all the things you need to make. It—wouldn't be fair, otherwise. I don't want you feeling like you're losing something every minute that goes by that you haven't started yet. I don't want—however long it takes me to decide if I'm letting you try it, even if it's a hundred years, I don't want that time measured in lives lost."
"And I might take Viasarae's suggestion and arrange for a bunch of people to have the power to raise the dead without me, although I guess that's less urgent when no one is dying anymore anyway..."
He sighs.
"But I'm getting ahead of myself. How does a thinking machine make everything good? And could it do the same things here even though there's magic? Magic, as you might have noticed, gives people a lot of power, and some of us decide to use that power to do things that definitely aren't good."
"...so there's two possibilities here, right. Either it can't stop people from misusing magic, and people will keep doing that, and everything will be less good than it is for you although maybe still better than it is for most people here... or it can stop people from misusing magic, which means—for one thing that it can stop me from misusing magic, and I'm not keen on that outcome—but also, and more importantly, that once I invite it to my empire I no longer have control of what happens here. And if it decides, someday, for whatever reason, to make things bad instead of good, I cannot stop it except maybe by melting the planet and starting over from scratch. And that's—" He shakes his head. "I don't like that. I want to know that things can't get any worse for my empire than I'm willing to let them."
"...this is so hard to explain. I'm not a superintelligence! I accordingly can't predict what a superintelligence would do! But this is just - everything you just said is the wrong way to be trying to guess what a superintelligence would do -
- hang on, I don't actually speak this language and need to talk to my thinking machine to come up with a good way to explain..."
"Imagine you are a bee," she says. "A talking bee for some reason I guess, and a human comes along and says, 'would you like to come live on my farm, it's full of fruit trees, I can keep you warm in the winter'. And you say 'well, logically, either you can stop us from stinging, or you can't stop us from stinging, and if you can't, then we will continue to have all the problems we have now where sometimes we sting each other and some bees die forever, and if you can, then we won't be able to sting you if you mess up, and then we won't be able to keep ourselves safe from predators'. And the human says 'uh, I can keep the predators out with a fence'. And you say, 'maybe, but what if you decide one day that you really want a bear instead of a beehive? Furthermore, we sting each other, to punish our fellow bees who do things we don't like' and the human says 'but I'm not going to want a bear on my fruit farm, and also, have you considered that those bees doing things you don't like could just live in a different hive, I have lots, I have a million' and you say 'I just don't know if the flowers you have are that good and it makes me very scared to imagine not being able to sting' and the human says 'okay for one thing if I were super scared of being stung I wouldn't even be a beekeeper, now, would I' and -
- can you just kind of get the sense of how this is a stupid conversation -"
"...I see what you're saying but I don't actually think that makes this conversation stupid, because—the bee is right that the human is asking him to put himself in the hands of something more powerful than he is, something he has no hope of defending himself from. When a bee and a human fight, you get a mildly inconvenienced human and a dead bee. Right now, in this world, I'm the incomprehensibly powerful thing that no one else has a hope of defending themselves against. And I do, actually, care a lot about the well-being of my people, so I've spent five thousand years getting better and better at making my empire a good place to live. But still, if I went and invited a hypothetical bee to live here, and they were worried I might hurt them... well, they'd be absolutely right, and if I really wanted that bee for some reason, it'd be up to me to find a way to reassure them about that. The tricky part here is that you're not the beekeeper, you're another bee who happened to wander into my field, and your ability to make assurances on behalf of your beekeeper is pretty limited."
"Sing can do very elaborate promise-verification stuff with other things like itself, it did that when there used to be more of them, but you're not a thing like itself to begin with and it isn't even here yet. I don't know what it will do because it's smarter than me but it will do something smart, okay, it will find the best thing to do and do that."
"I could imagine a different world run by another one of me who, I don't know, took up gardening instead of the things I like to do with my time, and had better luck in the magic department, and wants to share, and I can imagine how he'd handle this sort of thing—for that matter I can imagine a world run by another one of me who had worse luck in the magic department, and how I'd want to handle making contact with him—but your Sing isn't another one of me, and I don't know what its definition of the best thing is. And I don't know whether it'll think of me as someone it makes sense to negotiate with, or as... just another bee."
Another pause to gather his thoughts.
"Like—the way I'd handle this with myself, or with someone enough like myself that we'd both know this was going to work—the one with more power and prosperity says 'you can keep your empire, and I'll give you all the help I can reasonably give you without stepping too hard on your territory, and I won't take advantage of my power over you even if I'm very tempted to, as long as you respect my territory just as much.' And the one with less power and prosperity says 'all right, that makes sense, and I'll help you out as much as I reasonably can wherever my magic has an advantage over yours, because I want both of us to benefit from this'. And then we're set, and it doesn't matter that this conversation took place on opposite sides of a closed door because we each know the other well enough to guess what they'd say, and because—if one of me arrives here from another world holding his half of a deal like that, and he's right that I would've wanted to access his world if I'd known it was there, then of course I agree to it, because—if I hadn't been going to agree to it then he wouldn't have come. I'm not sure how much sense that makes outside my head, though. And I'm not sure I can make guesses like that about a thinking machine that has every reason to expect me to be basically a bee from its perspective."
"Yeah. The real trick would be—if I could make a good enough guess about what it might want and what it might be willing to agree to, if I could—show up with my half of a deal, and be confident that it would accept—but, like, I don't know that it would accept if, say, it thought it might have been able to get here without my help."
He thinks this over for a few seconds, then adds, "—the reason why I keep talking about it like I'm going to go visit your world is because I'm pretty sure that even if I can't get there on my own anytime soon, anything smart enough that it can make everything good for everyone all the time just by being smart at them will be able to figure it out for me. And—I like the look of this much better if I can go to your world and see if resurrection works there. It... feels less like being a bee, if there's something concrete I can contribute that it couldn't manage on its own, and I don't want your world to have to go without resurrection forever when instead I could be helping. I like helping! —and then of course I wonder if I could get a better hypothetical deal if I wasn't so eager to help out for its own sake, but—"
He has to pause another second to straighten his words out.
"...there's a thing where... something that really does want to make everything good, it's not going to look at me over here trying my best to think my way through what to do about this situation—I mean, not that it knows I'm here, but if it did—it's not going to look at that and think 'oh, yeah, it'd be a great idea to punish him for being the sort of person who'd help me just for the sake of helping', I'm pretty sure? Because if I can see the problem with that then clearly it can too."
"It's not going to punish you for helping. I... also don't think it will... reward you for being powerful, or for having particularly tricky tastes to navigate, or the combination... so I guess there's a sense in which the holding out for a good deal could theoretically make sense... except if you think you can get to my world I want you to do that right now so it can send a ton of robots here and I can go home to my girlfriend instead of inventing computers."
"I would love to send you home to your girlfriend and will happily try it as soon as I'm convinced that this doesn't end with my empire in the hands of the kind of beekeeper who doesn't mind crushing the occasional fuzzy friend to get at the honey. Like—not that I'm any better in that sense, really—but I know what to expect from myself, and if I look at my empire and decide I'm not doing a good enough job with it I can fix that, and—I have a responsibility to my people and that doesn't just go away the moment I hand them over to the, uh, 'robots'. Should I be asking what a robot is or will I not understand the answer? And would you be having an easier time with this conversation if I used translation magic?"
"Sure, just a minute."
He concentrates for a bit, and then ribbons of golden light dart through the air to connect the three of them, shining brightly for a moment before fading away into nothingness.
"There, that should be better," he says, and the meanings of the words and how they fit together into the sentence are as plainly understandable as if she'd been hearing this language all her life. "You can speak whatever language you like and I still won't know what a robot is but I'll at least have a vague idea of what kind of thing you might mean by it."
"I'll be obliged if you can speak the one we were speaking before so Page - that's my thinking machine, I like to call the general kind 'spirit guides' - can understand you," she says in English. "A robot is a machine that can move around and is directed by a thinking machine and a computer is the part that does the thinking."
"I am not really expecting it to be particularly accommodating about that," he says. "I sort of figure that if I want to torture people and there is no one around who wants to be tortured and an incomprehensibly powerful thing wants things to be good for everybody then I am just out of luck and will have to deal with that on my own time. It's—I'm not going to say I'm happy about it but there are, actually, things that are more important than me getting to keep my fucked-up hobbies."
"Sing does a lot but it doesn't totally supplant everything of importance humans do," she says. "In particular it mostly doesn't talk to people and doesn't take on a lot of traditional roles of government. I'm friends with a Lord of Mars who... I have no idea if you'd get along with but it crossed my mind... and in addition to occasionally kidnapping people as a sort of recreational theatrical hobby so I can duel him about it and stuff, he handles an amount of stuff that people want to feel someone important weighed in on. I don't know if that's the kinda thing you mean though."
"In my imperial capacity I mostly... exist as somebody people can go to if something gets screwed up in a way nobody else can fix. I try my best not to touch my government under normal circumstances, it works better that way, but every so often somebody is trading in illegal slaves and using magic to cover it up, or there's a hurricane headed for the coast and I can go untangle it before it flattens a city, or a junior clerk on the useful side of the palace finds evidence that her boss is taking bribes for things he really shouldn't be and she doesn't know who she can trust but she knows enough to realize she can come to me with it and I won't let her get hurt while we're figuring it out..."
"Yeah, I think so too. I'm not sure I could stand to do it all the time, they take twelve hours apiece to pull together and I am not the sort of person who can work tirelessly at tedious things forever, but—it'd be nice, if people stopped dying and I could bring them back to life without worrying I was signing myself up for an impossible job. I'd probably like to do it a lot."
"I don't actually know what it will do, to be clear. Uh, also I said that before you said it was about it being a responsibility, I think I wouldn't have said that if you'd said that part first, but like, people who were heads of state before Sing did not get special privileges in my world except insofar as their subjects thought having a real live queen or whatever was great and they were going to keep doing it for fun."
"Oh, yeah, that makes sense. No, I—even when I imagine getting the best possible version of this deal, it still involves people getting to leave my empire if they want to, and if that leaves me without much of an empire it's my business to change that by making it a more inviting one. But... yeah. It's about responsibility. And even if I opened up a portal and the whole empire filed through it to go live with the beekeeper, I'd still feel like it was my responsibility to be sure they didn't come to harm there, at least not any worse than they would have if they'd never left. So it's... troubling, not knowing how to be sure of that."
"I mean, I grew up there. I pointed out to Visarae that it's not like Sing couldn't hide something from me if it wanted to, but there would have to be two mistakes in it, not one, for that to happen, because it wouldn't automatically want to hide something just because it wanted to do that thing."
"Yeah."
He sighs. Stands up. "I don't mean to be rude, I just think better when I'm not sitting still," he says as he starts pacing back and forth across the little island. "Probably before I decide for sure whether to let you do this I'm going to spend a while flying about it first."
Pace pace pace. "So. The beekeeper. There's—mm, how do I put this... so if it was a person, right, I could say 'you want my people to have nice things and I also want my people to have nice things and if our definitions of nice things are close enough I think we can work together on this'. But it's not, and it's... you make it sound like the kind of thing where I don't get to have a different definition of nice things than it does, like, intentionally stupid example, if it thinks the colour orange is bad for people, and I don't, I don't get to run an empire where things are allowed to be orange, orange is just banned? And that makes me want to hold out for a deal where I do get that, because—mm, I have this thing where I can't stand to let somebody else do a worse job of running my empire than I would, and I know right now compared to your beekeeper the job I'm doing running my empire is garbage, but it doesn't have to be, I just haven't been trying to run a paradise. Because instead of that I've been arranging things to make it convenient for me to hurt people. I realize that doesn't make me sound very well qualified to run a paradise."
"It... kind of doesn't, but for the sake of argument," she says. "Uh, some people do disagree with Sing on what's good for people. The last two AIs left standing were Sing and one called Sugardream, and Sugardream was still pretty good, we think it would have been all right if it had won, but it did some stuff differently - in particular Sing avoids talking to people and Sugardream made friends, if you can call it that - I mean, I say 'if you can call it that' because I grew up with Sing, and the understanding is it doesn't and Sugardream didn't have some properties you need to be real friends, but Sugardream's friends didn't think so and so Sing talks to them, it sort of pretends to be Sugardream for them only it's using pieces of Sugardream to do it so it's barely even pretending. That's less gross for AIs than it sounds, to be clear, it's like - it's quoting a book, as another book. Uh, and lots of people want things Sing doesn't like and tries to make really hard - it's pretty good at making sure people don't want to irretrievably die but that's still happened a few times and some people went far enough out of their way to manage it. And it doesn't watch people as much as would probably be convenient because some people don't like that. So like, it might think orange is bad for people but it can probably understand if taking away everybody's orange is worse."
He nods thoughtfully through this explanation, paces, pauses at the table to grab a charming little bite-sized muffin, resumes pacing and fidgets with the muffin and eats it and thinks and speaks.
"I guess maybe the thing that I want is—in case there's something that's better for people than Sing, in case it's got a blind spot somewhere less obvious than the colour orange, or it's hiding something you don't know about, or there's something else going on that I haven't thought of—I want there to be room to notice that and do something about it. Because—that's the thing, right, about being the Emperor, that's what my job comes down to, is that I leave the business of governing to people who are better at it than me, but when somebody's screwing up, I notice and fix it. That's the thing I don't want to give up. The rest of it—I'm gonna have a time convincing myself to give up the slaves but I don't, actually, have any doubts about how that conversation is going to turn out, because—I don't know, I can't think of a way to make it make sense outside my head, it's just obviously true that with paradise on the horizon it stops mattering how personally uncomfortable I am about what I have to do to get there. And—I guess you could say, how good a paradise can it be, if people like me have to give things up to fit—but I think that's the wrong question, because... it's not a surprise to me, that when I hurt someone, their life would be better without me in it. It's always been a worthwhile trade but it is a trade, my happiness for theirs, and if you care about everyone equally it's very obviously a terrible deal, and even leaving that aside there's things I'm giving up by living this way."
She has a muffin herself while he's talking. "I... don't really know what to do about that," she says. "If it wants to hide something it'll just succeed, it's more than smart enough. Uh, there's a technical term in AI theory, 'corrigible', where you have an AI that forms its opinions but if a human or possibly just some specific human says 'no, you've made a mistake', it assumes the human is right and it's wrong. Sing's not corrigible, it won't just decide you're right if you disagree with it on something, but it does still think it's information when people disagree with it? It pays attention."
"Well, the premise of this conversation is that I couldn't if I tried, right? And anyway I'm not—I've been doing this for five thousand years but that's not long enough to know how to run a paradise, not really. Not the way I've been using it. And—if I have any hope of arguing that I get to keep my own territory, I can't be planning to take Sing's, that'd be—stupid."
He pauses at the edge of the island, looking out at the sky.
"...you're not wrong that I'd have the impulse, though."
"...yeah," he says, turning back to cross the island again, "like I said. I don't know how to run a paradise yet."
Pace pace pace.
"I think... I want to, though. I want to learn how to build a world that's good enough that yours isn't obviously better anymore. And—I don't know, maybe Sing'll let me do that. It'd definitely be stupid and unfair to wait to send you home until I figure it out by myself."
"And anyway, even if it wasn't stupid and unfair to you in particular, it's—making my people wait another several thousand years for me to get my shit together before they can have their paradise. I'm not gonna do that. I just—wish it wasn't a matter of whether Sing will let me."
"...I'm pretty sure there isn't actually anything it can do that changes the thing where it's more powerful than me and has good reasons not to let me do whatever I want. I'm pretty sure that's—the sort of thing where there's no getting around it with cleverness, I just am the sort of person who doesn't like to be outmatched and doesn't like to be answerable to any authority outside my own, and however many of those there are in my world right now, I'm the only one who gets to be satisfied on that score at the moment, and soon enough I won't be either, and it's worth it to give my people their paradise but that doesn't make it not sting."
"It lets people mostly alone as long as they're not dying. There's a religious minority called the Amish who are against technology in general, they still do, like, farming, and Sing doesn't let them die forever, it insists on freezing them, but otherwise they can farm if they want. I guess it's probably doing something to make sure they don't abuse their children but I don't have the slightest idea what, it's really backed off."