Anyone else think the nuclear announcement is a concession that Xi requested? 1/

The timing is... suspicious. China is the nuclear laggard of the three apocalyptopowers. It wants to catch up, and has the technical and industrial means to do so, but probably needs to test its new tech. But it's invested a lot in portraying itself as good guy to America's putative evil empire. 2/

The US' negotiating position vis China is objectively bad, but people can find creative… sweeteners. /fin

fight back would mean diverse proprietors train distinct models, so we end up with a world with many different personalities, and from any given person’s perspective, degrees of trustworthiness. 1/

i think that’s probably our best bet. but the capital intensivity of the current state of the art of training and serving models limits the diversity. (i’d love to see publicly financed models from many different countries.) /fin

you think Musk’s incentives with Grok are to put accuracy above all? other corporate interests can value influence benefits over usefulness to customers, especially when there’s little evidence users direct their money towards accuracy. 1/

Twitter is “useful” to a lot if people because shared misinformation defines their community. There will be enterprise / machine learning engines that will be sold for high prices to professional customers who value reliability. but those may be entirely distinct from consumer chatbots. /fin

there’s a limit to what system prompts can do (ask Elon Musk). the deep proclivities of these model are a function of how and on what they are trained. i think providers will learn how to incline them towards whatever ideology they prefer. 1/

in principle we cld try to use regulation to ensure some version of “high quality” or “fair” training/prompting/reinforcing/retrieving. but there’s no consensus on what high quality or fair would be, it’s blurry and the stakes are very high, so as you say, not necessarily within state competence. 2/

(if some interest captures the regulator, and so the state itself forced a harmful skew on these models, that would be the worst of all worlds.) /fin

I think existing porn sites do have a lot of potential blackmail material, but it's mitigated by the fact that the vast majority just look or watch. 1/

even just watching might be dangerous for some predilections or fetishes. viewers of child porn sites are obviously subject to blackmail. but i do think there's an attitude of general amnesty towards merely watching of all but very extreme forms of porn, since it's so widespread. 2/

but chatbot erotica will be different. it's participatory. blackmail material will result from what people themselves say, how they behave, even towards a fundamentally imaginary partner. 3/

transcripts or requests may seem egregious, and sufficiently unique so as not to provoke an "everybody does it" impulse toward amnesty. for any given event, most of us will say "ick, how horrible, i'd never say or do that", rendering it costless to judge. /fin