• 3 Posts
  • 48 Comments
Joined 6 months ago
cake
Cake day: August 27th, 2025

help-circle


  • It certainly seems to be trending in that direction, no? A lot of the “best” ideas do tend to get crowbarred out of the US.

    OTOH, with the EU pushing to divest itself of American software and policies, perhaps there’s still some wriggle room.

    OTOOH, because of the nature of the Fediverse, something like this can happen in theory:

    • You use a German-hosted instance → primarily subject to EU/GDPR
    • Your posts replicate to instances in other countries (including the US), which you don’t directly control (unless you self-host and block that)
    • Those servers operate under their own local laws
    • End result: your reposted data and meta data may now fall within the American legal domain.













  • re: the KB tool, why not just skip the llm and do two chained fuzzy finds? (what knowledge base & question keywords)

    Yep, good question. You can do that, it’s not wrong. If your KB is small + your question is basically “find me the paragraph that contains X,” then yeah: two-pass fuzzy find will dunk on any LLM for speed and correctness.

    But the reason I put an LLM in the loop is: retrieval isn’t the hard part. Synthesis + constraint are. What a LLM is doing in KB mode (basically) is this -

    1. Turns question into extraction task. Instead of “search keywords,” it’s: “given these snippets, answer only what is directly supported, and list what’s missing.”

    2. Then, rather that giving 6 fragments across multiple files, the LLM assembles the whole thing into a single answer, while staying source locked (and refusing fragments that don’t contain the needed fact).

    3. Finally: it has “structured refusal” baked in. IOW, the whole point is that the LLM is forced to say “here are the facts I saw, and this is what I can’t answer from those facts”.

    TL;DR: fuzzy search gets you where the info lives. This gets you what you can safely claim from it, plus an explicit “missing list”.

    For pure retreval: yeah - search. In fact, maybe I should bake in a >>grep or >>find commands. That would be the right trick for “show me the passage” not “answer the question”.

    I hope that makes sense?


  • Responding to my own top post like a FB boomer: May I make one request?

    If you found this little curio interesting at all, please share in the places you go.

    And especially, if you’re on Reddit, where normies go.

    I use to post heavily on there, but then Reddit did a reddit and I’m done with it.

    https://lemmy.world/post/41398418/21528414

    Much as I love Lemmy and HN, they’re not exactly normcore, and I’d like to put this into the hands of people :)

    PS: I am think of taking some of the questions you all asked me here (de-identified) and writing a “Q&A_with_drBobbyLLM.md” and sticking it on the repo. It might explain some common concerns.

    And, If nothing else, it might be mildly amusing.


  • Yes of course.

    Abliterated is a technical LLM term meaning “safety refusals removed”.

    Basically, abliteration removes the security theatre that gets baked into LLM like chatGPT.

    I don’t like my tools deciding for me what I can and cannot do with them.

    I decide.

    Anyway, the model I use has been modified with a newer, less lobotomy inducing version of abliteration (which previously was a risk).

    https://huggingface.co/DavidAU/Qwen3-4B-Hivemind-Instruct-NEO-MAX-Imatrix-GGUF/tree/main

    According to validation I’ve seen online (and of course, I tested it myself), it’s lost next to zero “IQ” and dropped refusals by about…90%.

    BEFORE: Initial refusals: 99/100

    AFTER: Refusals: 8/100 [lower is better], KL divergence: 0.02 (less than 1 is great, “0” is perfect.)

    In fact, in some domains it’s actually a touch smarter, because it doesn’t try to give you “perfect” model answers. Maths reasoning for example, where the answer is basically impossible, it will say “the answer is impossible. Here’s the nearest workable solution based on context” instead of getting stuck in a self-reinforcing loop, trying to please you, and then crashing.

    In theory, that means you could ask it for directions on how to cook Meth and it would tell you.

    I’m fairly certain the devs didn’t add the instructions for that in there, but if they did, the LLM won’t “sorry, I can’t tell you, Dave”.

    Bonus: with my harness over the top, you’d have an even better idea if it was full of shit (it probably would be, because, again, I’m pretty sure they don’t train LLM on Breaking Bad).

    Extra double bonus: If you fed it exact instructions for cooking meth, using the methods I outlined? It will tell you exactly how to cook Meth, 100% of the time.

    Say…you…uh…wanna cook some meth? :P

    PS: if you’re more of a visual learner, this might be a better explanation

    https://www.youtube.com/watch?v=gr5nl3P4nyM


  • It’s copyLEFT (AGPL-3.0 license). That means, free to share, copy, modify…but you can’t roll a closed source version of it and sell it for profit.

    In any case, I didn’t build this to get rich (fuck! I knew I forgot something).

    I built this to try to unfuck the situation / help people like me.

    I don’t want anything for it. Just maybe a fist bump and an occasional “thanks dude. This shit works amazing”




  • For the record: none of my posts here are AI-generated. The only model output in this thread is in clearly labeled, cited examples.

    I built a tool to make LLMs ground their answers and refuse without sources, not to replace anyone’s voice or thinking.

    If it’s useful to you, great. If not, that’s fine too - but let’s keep the discussion about what the system actually does.

    Also, being told my writing “sounds like a machine” lands badly, especially as an ND person, so I’d prefer we stick to the technical critique.