fight back would mean diverse proprietors train distinct models, so we end up with a world with many different personalities, and from any given person’s perspective, degrees of trustworthiness. 1/

i think that’s probably our best bet. but the capital intensivity of the current state of the art of training and serving models limits the diversity. (i’d love to see publicly financed models from many different countries.) /fin

you think Musk’s incentives with Grok are to put accuracy above all? other corporate interests can value influence benefits over usefulness to customers, especially when there’s little evidence users direct their money towards accuracy. 1/

Twitter is “useful” to a lot if people because shared misinformation defines their community. There will be enterprise / machine learning engines that will be sold for high prices to professional customers who value reliability. but those may be entirely distinct from consumer chatbots. /fin

there’s a limit to what system prompts can do (ask Elon Musk). the deep proclivities of these model are a function of how and on what they are trained. i think providers will learn how to incline them towards whatever ideology they prefer. 1/

in principle we cld try to use regulation to ensure some version of “high quality” or “fair” training/prompting/reinforcing/retrieving. but there’s no consensus on what high quality or fair would be, it’s blurry and the stakes are very high, so as you say, not necessarily within state competence. 2/

(if some interest captures the regulator, and so the state itself forced a harmful skew on these models, that would be the worst of all worlds.) /fin

I think existing porn sites do have a lot of potential blackmail material, but it's mitigated by the fact that the vast majority just look or watch. 1/

even just watching might be dangerous for some predilections or fetishes. viewers of child porn sites are obviously subject to blackmail. but i do think there's an attitude of general amnesty towards merely watching of all but very extreme forms of porn, since it's so widespread. 2/

but chatbot erotica will be different. it's participatory. blackmail material will result from what people themselves say, how they behave, even towards a fundamentally imaginary partner. 3/

transcripts or requests may seem egregious, and sufficiently unique so as not to provoke an "everybody does it" impulse toward amnesty. for any given event, most of us will say "ick, how horrible, i'd never say or do that", rendering it costless to judge. /fin

for now, chatbots tend to sycophantically confabulate upon and reinforce user prejudices and inclinations, rather than reining them in. there’ve been prominent cases of people going a bit mad this way. 1/

that could be remedied. chatbots could have some version of consensus reality towards which they guide users. but then these massive, centralized, profit-seeking companies that run the chatbots would largely define that “consensus”. 2/

do you think “touching grass” with that version of reality would, eg, dissuade people from voting for Donald Trump, or, less politically, force them to confront realities like climate change is real and vaccines work and tylenol has not been shown to cause autism? /fin

Fake incriminating material is already fully democratized. This would be authentic. 1/

I suspect the big AI platforms will include authentication tokens in the data they collect (something like automated hashes published to a blockchain L2, establishing proof of existence with a firm-controlled timestamp and signature), distinguishing it from unauthenticated kompromat. /fin