• 15 Posts
  • 307 Comments
Joined 3 years ago
cake
Cake day: June 17th, 2023

help-circle
  • There are better alternatives, podman is daemonless and rootless by default, comes with a docker-compatible CLI, and far better container network implementations. It is also provided as stable/LTS package in Debian repositories so you won’t have to upgrade your container runtime every 2 days (causing downtime), like running the upstream Docker package does.

    The only reason to keep using Docker nowadays is if you have a lot of legacy apps that depend on Docker-specific features (e.g. require rootful containers). For most workflows it is a matter of alias docker=podman. If you use docker-compose, you do need to port your setup to podman quadlets, or systemd-managed containers though. For me it was worth it.


    • Small 4B models like gemma3 will run on anything (I have it running on a 2020 laptop with integrated graphics). Don’t expect superintelligence, but it works for basic classification tasks, writing/reviewing/fixing small scripts and basic chat, writing, etc
    • I use https://github.com/ggml-org/llama.cpp in server mode pointing to a directory of GGUF model files downloaded from huggingface. I access it it from the built-in web interface or API (wrote a small assistant script)
    • To load larger models you need more RAM (preferably fast VRAM/GPU but DDR5 on the motherboard will work - it will be noticeably slower). My gaming rig with 16GB AMD 9070 runs 20-30B models at decent speeds. You can grab quantized (lower precision, lower output quality) versions of those larger models if the full-size/unquantized models don’t fit. Check out https://whatmodelscanirun.com/
    • For image generation I found https://github.com/vladmandic/sdnext which works extremely well and fast wth Z-Image Turbo, FLUX.1-schnell, Stable Diffusion XL and a few other models

    As for the prices… well the rig I bought for ~1500€ in september is now up to ~2200€ (once-in-a-decade investment). It’s not a beast but it works, the primary use case was general computing and gaming, I’m glad it works for local AI, but costs for a dedicated, performant AI rig are ridiculously high right now. It’s not economically competitive yet against commercial LLM services for complex tasks, but that’s not the point. Check https://old.reddit.com/r/LocalLLaMA/ (yeah reddit I know). 10k€ of hardware to run ~200-300B models, not counting electricity bills




  • I’m in the same boat, running a Gitlab Mattermost instance for a small team.

    Gitlab has not announced yet what will happen with the bundled Mattermost, but I guess it will be dropped entirely, or be hit by the new limitations (what will hit us the hardest is the 10000-most-recent messages limitation, anything further than that will be hidden behind a paywall - including messages sent before the new limitations come in effect - borderline ransomware if you ask me)

    I know there are forks that remove the limitation, may end up doing that if the migration path is not too rough.

    I used to run a Rocket.Chat instance for another org, became open-core bullshit as well. I’m done with this stuff.

    I have a small, non-federated personal Matrix + Element instance that barely gets any use (but allows me to get a feeling of what it can do) - I don’t like it one bit. The tech stack is weird, the Element frontend receives constant updates/new releases that are painful to keep up with, and more importantly, UX is confusing and bad.

    So I think I’ll end up switching this one for a XMPP server. Haven’t decided which one or which components around it precisely. I used to run prosody with thick clients a whiiille ago and it was OK. Nextcloud Talk might also work.

    My needs are simple, group channels, 1-to-1 chat, posting files to a channel. ideally temporary many-to-many chats, decent web UI.

    Voice capabilities would be a bonus (I run and use a mumble server and it absolutely rules once you’ve configured the client, but it doesn’t integrate properly into anything else, and no web UI), as well as some kind of integration with my Jitsi Meet instance. E2E encryption nice but not mandatory. Semi-decent mobile clients would be nice.

    For now, wait and see.








  • A full-blown samba domain is extremely overkill if you don’t have a fleet of windows machines.

    You can get centralized user management with a simple LDAP server or similar, no need for a domain.

    Also, snapshots-based backups have limited uses (can’t easily restore only a single file, eats quite a bit of storage). The only times where I actually needed backups were because I fucked up a single application or database, don’t want to rollback the whole OS/data drive for that.


  • https://lemmy.world/post/34029848/18647964

    • Hypervisor: Debian stable + libvirt or PVE if you need clustering/HA
    • VMs: Debian stable
    • podman if you need containerization below that

    You can migrate VMs live between hosts (it’s a bit more work if you pick libvirt, but the overhead/features or proxmox are sometimes overkill, libvirt is a bit more barebones, each has its uses), have a cluster-wide L2 network, use a machine as backup storage for others… use VM snapshots for rollback, etc. Regardless of containerization/orchestration below that, a full hypervisor is still nice to have.

    I deploy my services directly to the VM or as podman containers in said VMs. I use ansible for all automation/provisioning (though there are still a few basic provisioning/management to bootstrap new VMs, if it works it works)







  • If you needs are simple, write a simple playbook using the proxmox ansible module https://docs.ansible.com/ansible/latest/collections/community/general/proxmox_kvm_module.html

    Terraform/Opentofu provides more advanced stuff but then you have to worry about persistent state storage, the clunky DSL… used it when acsolutely needed, you can do 90% of this stuff with the proxmox ansible module.

    If you need to make your playbook less verbose, move the logic to a role so that you can configure your VMs from a few lines in the playbook/host_vars. Mine looks like this (it’s for libvirt and not proxmox, but the logic is the same)

    # playbook.yml
    - hosts: hypervisor.example.org
      roles:
        - libvirt
    
    # host_vars/hypervisor.example.org.yml
    libvirt_vms:
      - name: vm1.example.org
        xml_file: "{{ playbook_dir }}/data/libvirt/vm1.example.org.xml"
        state: running
        autostart: yes
      - name: vm2.example.org
        xml_file: "{{ playbook_dir }}/data/libvirt/vm2.example.org.xml"
        autostart: no
      - name: vm3.example.org
        xml_file: "{{ playbook_dir }}/data/libvirt/vm3.example.org.xml"
        autostart: no
      - name: vm4.example.org
        xml_file: "{{ playbook_dir }}/data/libvirt/vm4.example.org.xml"
        autostart: no
        disk_size: 100G
    



    • Ever tested restoring those backups? Do you have the exact procedure written down? Does it still work? If the service gets compromised/data corrupted on sunday, and your backup runs, do you still have a non-compromised backup and how old is it?
    • How timely can you deal with security fixes, and how will you be alerted that a security fix is available?
    • How do you monitor your services for resource availability, errors in logs, security events?
    • How much downtime is acceptable for routine maintenance, and for incidents?
    • Do you have tooling to ensure you can redeploy the exact same configuration to another host?
    • How do you test upgrades before pushing them to production?

    Not saying this is impossible, you just need to have these questions in mind, and the answers written down before you start charging people for the service, and have the support infrastructure ready.

    Or you can just provide the service for free, best-effort without guarantees.

    I do both (free services for a few friends, paid by customers at $work, small team). Most of the time it’s smooth riding but it needs preparation (and more than 1 guy to handle emergencies - vacations, bus factor and all that).

    For the git service I can recommend gitea + gitea-actions (I run the runners in podman). Gitlab has more features but it can be overwhelming if you don’t need them, and it requires more resources.




  • vegetaaaaaaa@lemmy.worldtoSelfhosted@lemmy.worldVersion Dashboard
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    10 months ago

    I use RSS feeds, bump version numbers when a new release is out, git commit/push and the CI does the rest (or I’ll run the ansible playbook manually).

    I do check the release notes for breaking changes, and sometimes hold back updates for some time (days/weeks) when the release affects a “critical” feature, or when config tweaks are needed, and/or run these against a testing/staging environment first.