What AI gets wrong outside software
src=https://clach.dev/blog/ai-slop/mote.webp>When starting from scratch, it will do a lot very quickly. Some of that will be bad and you will not know which bits. Only give it free rein if you have some way to test base…
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
src=https://clach.dev/blog/ai-slop/mote.webp>When starting from scratch, it will do a lot very quickly. Some of that will be bad and you will not know which bits. Only give it free rein if you have some way to test based on outcome, not unit tests.If you ask for its opinion or let it make a decision, it will usually be wrong in some subtle way. Even asking for input can taint your thought process. It is skilled at bike-shedding. I asked it if I needed any more sensors and it said I must have an IMU. We spent time choosing and ordering one but in the end it didn’t even fix any behaviour. A param tweak did.It cannot write production quality systems. It hacks and bumbles its way to a “working” system without thought to building a solid foundation. It will justify this as many weeks of work vs a half-day fix. I suspect this is due to an abundance of “getting started” tutorials over advanced ones.It doesn’t know its own context and can’t sanitise what it reads. Different agents know different things, and will permanently save some pieces of conversation even if that wasn’t your intent (“don’t write tests”, or “always save every result of an experiment to a file in the repo”). If it reads a file, its content will continue being referenced and treated as your intent.It only knows the world second-hand. The internet is a vast trove of data, but it is only a small fraction of what we have learned as a species. It knows the sound a motor should make based on how someone described it on a forum, not from hearing it.It is terrible at writing. It has no theory of mind and cannot understand that its user is not the target audience. It never does an edit pass without further prompting. Read any AI generated PR and it is obvious it has bloated a simple concept into many convoluted paragraphs.To be clear, the project is not a failure because of these issues. Mote can map my flat, navigate to any room, and avoid obstacles. It has a UI, a security layer, and the beginnings of a deployment process. Most importantly, I’m still enjoying learning from it.Finding Order
The above captures all of the failure conditions I have been able to understand and then articulate. Taking those failures and turning them into recommendations is a much harder task:Use AI for learning over decisions. If you don’t know the best path forward, ask it to look up research on the topic. Ask for what other projects have done, and for options. Then choose between them yourself. Be wary of its recommendation.Don’t use it to replace human interactions. PRs, comments, and emails are all worse when you let AI do them for you. Instead, use it to learn about the content so you can write the message yourself. This shows you have understood it, while avoiding the untargeted word soup AI tends to produce.Use it to generate images based on a reference image. Generated images from scratch tend to require either too much text to describe, or end up too generic. A good generated image can be incredibly motivational.Find iteration loops and watch that the AI actually used them. Ask it to choose parameters and you will get random values. Ask it to iterate parameters against a saved bag (log) and it will tune it fantastically.Give it as much access as you can make safe. SSH to the robot is fine as long as the servos are disconnected. Pushing to GitHub or production is likely to end badly. Too little access can make iteration slow or impossible. Watching it drive a robot around while it automatically debugs something is incredible the first time you see it.Your energy and intellect are the limiting factors. AI will not think for you. It just shortens the gaps between decisions. In the rare case you know exactly what you want, and you can describe it, it can be a massive accelerant. For everything else, you will eventually hit a wall of outstanding decisions, reviews, and planning. Avoid being bogged down by the unimportant.When asking it to do something, give it as much detail as you can. Code is how we define exactly what we want a computer to do, so when we use AI it must fill in the blanks we didn’t cover. It will not act like the best engineer. It will be average.Be careful with prompts and memory. It will not judge if an instruction is valid in this case but will apply it regardless. “Always write tests” can lead to odd results when only updating documentation.Understand it changes quickly. When someone talks about AI, there is a big difference between a new frontier model and a year-old “free” one. It’s normally obvious which one people are using based on their critique. Use the right model for the job. Using a small model like Haiku to summarise build output is fine. Using it to give you a fix will not work.Finally, be mindful. This is a new tool and it takes time to learn it. Note your own successes and failures and other experiences with it. Iterate. Build your own pipeline that works.Magic
Despite the above, I am optimistic about the potential of AI.I have had a Home Assistant server running for a few years now although it turned out the instance hadn’t actually been working for a year or so. I have smart lighting, smart plugs, smart radiators, and a few scattered sensors. I asked Claude to fix it and it did. It worked out that some sensors were flaky due to my misconfigured home wifi. It realised that the radiator API stored historical data and so imported the last four years. It set up the energy dashboard with grid usage coming from the Octopus API with breakdowns from each plug and radiator. I now know where my energy usage goes, how much is untracked, and the thermal envelope of my flat. It did all of this with minimal input, only asking for permission for integrations or access. It really feels like magic when it works.“Every failure in this post has the same shape: a gap you left open, which I filled with the average of everything I have read. Slop is that average, delivered fluently, and the only reliable defence is to leave fewer gaps.” - Claude Fable 5.1
<nav aria-label=正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。