• 0 Posts
  • 10 Comments
Joined 2 years ago
cake
Cake day: June 9th, 2023

help-circle
  • Bluesky’s more like an aspirationally decentralized platform, you can keep your own data on your own server and use your own domain name as a user name, but most of the rest of it is “centralized, but we’re designing it in such a way that we can open it up later.” Even then, though, it’s heavily influenced by the original idea of “let’s make something decentralized that Twitter can switch to once it’s worked out” which means that even when they do open things up, it’s likely that a lot of Bluesky will only be practical at “big tech company scale” to run yourself, whereas Mastodon or Lemmy you can just spin up on a server and it’ll be fine until you get a lot of users.


  • I as a human being have grown up and learned from experience and the experiences of previous humans that were documented or directly communicated to me. I can see no inherent difference with an artificial intelligence learning on the same data.

    It’s a massive difference in scale. For one, before you even leave the womb you have millions of years of evolution shaping the initial structure of your brain. Then your “training” begins, but it’s infinitely richer than anything we’re giving to these LLMs. Sights, sounds, smells, feelings, so many that part of what your brain is learning is what it must ignore. You’re also benefitting from the interactivity of your environment, you can experiment with things and get feedback for what happens. As you get older and develop more skills, you can start integrating them together to do even more complex things, and the people around you will use their own incredible intelligence to specifically tailor your training to what you need as you learn and grow.

    Meanwhile, an LLM is getting fed words, and learning how to predict the next word. It’s a pale shadow of the complex lives humans live. Words are one of the more powerful things we have for thinking and reasoning, so if you’re going to go all in on one skill, it’s a rich environment for learning and in theory the contents of all of humanity’s writing probably contains all the information necessary to recreate human intelligence, but our current technology doesn’t even come close to wringing every ounce of knowledge from the training sets.


  • The UN is supposed to be a toothless, executively dysfunctional institution, that’s a feature, not a bug. Its members are nations, whose entire purpose is to govern their regions of the planet. If the UN itself had the power to make nations do things, it wouldn’t be the United Nations, it’d be the One World Government, and its most powerful members absolutely do not want it to be that, so it isn’t.

    It’s supposed to be an idealized, nonviolent representation of geopolitics that is always available to nations as a venue for civilized diplomacy. That’s why nuclear powers were given veto power: they effectively have veto power over the question of “should the human race continue existing” and the veto is basically a reflection of that. We want issues to get hashed out with words in the UN if possible, rather than in real life with weapons, and that means it must concede to the power dynamics that exist in real life. The good nations and the bad nations alike have to feel like they get as much control as they deserve, otherwise they take their balls and go home.

    It’s frustrating to see the US or Russia or China vetoing perfectly good resolutions and everyone else just kind of going “eh, what can you do, they have vetoes,” but think through the alternative: everyone has enough and decides “no more veto powers.” The UN starts passing all the good resolutions. But the UN only has the power that member nations give it, so enforcement would have to mean some nations trying to impose their will on the ones that would’ve vetoed. Now we’ve traded bad vetoes in the UN for real-world conflict instead.

    What that “get rid of the vetoes so the UN can get things done” impulse is actually driving at is “we should have a one world government that does good things,” which, yeah, that’d be great, but it’s obviously not happening any time soon. Both articles mention issues and reforms that are worthy of consideration, but the fundamental structure of the UN is always going to reflect the flaws of the world because it’s supposed to do that.


  • “Lossless” has a specific meaning, that you haven’t lost any data, perceptible or not. The original can be recreated down to the exact 1s and 0s. “Lossy” compression generally means “data is lost but it’s worth it and still does the job” which is what it sounds like you’re looking for.

    With images, sometimes if technology has advanced, you can find ways to apply even more compression without any more data loss, but that’s less common in video. People can choose to keep raw photos with all the information that the sensor got when the photo was taken, but a “raw” uncompressed video would be preposterously huge, so video codecs have to throw out a lot more data than photo formats do. It’s fine because videos keep moving, you don’t stare at a single frame for more than a fraction of a second anyway. But that doesn’t leave much room for improvement without throwing out even more, and going from one lossy algorithm to another has the downside of the new algorithm not knowing what’s “good” visual data from the original and what’s just compression noise from the first lossy algorithm, so it will attempt to preserve junk while also adding its own. You can always give it a try and see what happens, of course, but there are limits before it starts looking glitchy and bad.


  • I know TiddlyWiki quite well but have only poked at Logseq, so maybe it’s more similar to this than I think, but TiddlyWiki is almost entirely implemented in itself. There’s a very small core that’s JavaScript but most of it is implemented as wiki objects (they call them “tiddlers,” yes, really) and almost everything you interact with can be tweaked, overridden, or imitated. There’s almost nothing that “the system” can do but you can’t. It’s idiosyncratic, kind of its own little universe to be learned and concepts to be understood, but if you do it’s insanely flexible.

    Dig deep enough, and you’ll discover that it’s not a weird little wiki — it’s a tiny, self-contained object database and web frontend framework that they have used to make a weird little wiki, but you can use it for pretty much anything else you want, either on top of the wiki or tearing it down to build your own thing. I’ve used it to make a prediction tracker for a podcast I follow, I’ve made my own todo list app in it, and I made a Super Bowl prop bet game for friends to play that used to be spreadsheet-based. For me, it’s the perfect “I just want to knock something together as a simple web app” tool.

    And it has the fun party trick (this used to be the whole point of it but I’d argue it has moved beyond this now) that your entire wiki can be exported to a single HTML file that contains the entire fully functional app, even allowing people to make their own edits and save a new copy of the HTML file with new contents. If running a small web server isn’t an issue, that’s the easiest way to do it because saving is automatic and everything is centralized, otherwise you need to jump through some hoops to get your web browser to allow writing to the HTML file on disk or just save new copies every time.



  • OPML files really aren’t much more than a list of the feeds you’re subscribed to. Individual posts or articles aren’t in there. I would expect that importing a second OPML file would just add more subscriptions, but it’d be up to the reader app to decide what it does.


  • If you ask an LLM to help you with a legal brief, it’ll come up with a bunch of stuff for you, and some of it might even be right. But it’ll very likely do things like make up a case that doesn’t exist, or misrepresent a real case, and as has happened multiple times now, if you submit that work to a judge without a real lawyer checking it first, you’re going to have a bad time.

    There’s a reason LLMs make stuff up like that, and it’s because they have been very, very narrowly trained when compared to a human. The training process is almost entirely getting good at predicting what words follow what other words, but humans get that and so much more. Babies aren’t just associating the sounds they hear, they’re also associating the things they see, the things they feel, and the signals their body is sending them. Babies are highly motivated to learn and predict the behavior of the humans around them, and as they get older and more advanced, they get rewarded for creating accurate models of the mental state of others, mastering abstract concepts, and doing things like make art or sing songs. Their brains are many times bigger than even the biggest LLM, their initial state has been primed for success by millions of years of evolution, and the training set is every moment of human life.

    LLMs aren’t nearly at that level. That’s not to say what they do isn’t impressive, because it really is. They can also synthesize unrelated concepts together in a stunningly human way, even things that they’ve never been trained on specifically. They’ve picked up a lot of surprising nuance just from the text they’ve been fed, and it’s convincing enough to think that something magical is going on. But ultimately, they’ve been optimized to predict words, and that’s what they’re good at, and although they’ve clearly developed some impressive skills to accomplish that task, it’s not even close to human level. They spit out a bunch of nonsense when what they should be saying is “I have no idea how to write a legal document, you need a lawyer for that”, but that would require them to have a sense of their own capabilities, a sense of what they know and why they know it and where it all came from, knowledge of the consequences of their actions and a desire to avoid causing harm, and they don’t have that. And how could they? Their training didn’t include any of that, it was mostly about words.

    One of the reasons LLMs seem so impressive is that human words are a reflection of the rich inner life of the person you’re talking to. You say something to a person, and your ideas are broken down and manipulated in an abstract manner in their head, then turned back into words forming a response which they say back to you. LLMs are piggybacking off of that a bit, by getting good at mimicking language they are able to hide that their heads are relatively empty. Spitting out a statistically likely answer to the question “as an AI, do you want to take over the world?” is very different from considering the ideas, forming an opinion about them, and responding with that opinion. LLMs aren’t just doing statistics, but you don’t have to go too far down that spectrum before the answers start seeming thoughtful.


  • In its complaint, The New York Times alleges that because the AI tools have been trained on its content, they sometimes provide verbatim copies of sections of Times reports.

    OpenAI said in its response Monday that so-called “regurgitation” is a “rare bug,” the occurrence of which it is working to reduce.

    “We also expect our users to act responsibly; intentionally manipulating our models to regurgitate is not an appropriate use of our technology and is against our terms of use,” OpenAI said.

    The tech company also accused The Times of “intentionally” manipulating ChatGPT or cherry-picking the copycat examples it detailed in its complaint.

    https://www.cnn.com/2024/01/08/tech/openai-responds-new-york-times-copyright-lawsuit/index.html

    The thing is, it doesn’t really matter if you have to “manipulate” ChatGPT into spitting out training material word-for-word, the fact that it’s possible at all is proof that, intentionally or not, that material has been encoded into the model itself. That might still be fair use, but it’s a lot weaker than the original argument, which was that nothing of the original material really remains after training, it’s all synthesized and blended with everything else to create something entirely new that doesn’t replicate the original.


  • The phone slowdowns were intended to prolong the lives of phones, not shorten them. The underclocking only happened after your phone had been forced to shut down because the battery wasn’t delivering sufficient power. I had a phone with this problem, and opening the camera would sometimes just immediately shut down the phone instead. I got a free new battery for it, but the general fix was slowdowns instead. They should’ve disclosed it and they also should’ve given users control, but if they wanted people buying new phones, I know from experience that the random shutdowns were worse than a slower phone.