I bought into the ecosystem while taking my networking cert classes back in 2017. They were much cheaper than Cisco gear for business-grade networking, and overall I've been happy with them. ...
This was one of the very first ZFS demos I've seen at some sales engineers training by Sun Microsystems Germany (Sun and Sun partner engineers), probably sometimes in 2005. Demo was to export the pool, unplug and scramble the thumb drives, plug all drives back in and import the pool. #vintagecomputing#zfs
I've been tuning my hybrid ZFS pool lately. Optane for metadata and small files, HDDs for bulk storage, and cleaned up my L2ARC. Things are finally starting to feel right.
There's one subject I haven't seen photographers talk about
The subject of photo archive management
It's not as easy as you may think.
Many people, especially in this last decade, grab an Android, grab a toy camera, as I call the small yet much handier cameras from camera brands like Fuji Nikon and Canon then point at the scene, often don't even know what the difference is between proper lighting and inverse lighting, then shoot their photograph.
Most end users don't even realize that they need an infrastructure to properly archive their photographs whatever the quality maybe.
The last time I lost data was in the floppy disk years; after that I never lost anything I archived on magnetic 🧲 media.
That case was a dirty read write head on a floppy drive which scratched my master and my backup disc on surface one.
I mitigated that error by using two floppy drives. One FDD specifically for the master disc after I cleaned the heads, one drive specifically for the backup disc after I cleaned those heads.
In the harddisk era I never lost any data. I use tried and tested procedures which are documented in request for comments RFC. You will find many, just search for them. Even those using tar -cvfz are good
At a certain point in time I switched from film to DSLR. The question was immediate
With my negatives I print what I need and since film 🎥 lasts {almost} forever I had no backup issues since everything was analog.
My digital DSLR body comes with two SD card slots. 8 GB per slot giving me 16 GB of storage. I usually shoot one card full then go to the other. I use my analog method of shooting only proper scenes. I don't use the camera as a machine gun and then sort out through all the mess which photographs are fair. Quality above quantity. I have a couple of hard drives where i stored the backups and the data Is never Lost. Sounds easy right?
When another DSLR or Point & Shoot body comes in the mix you know need to manage the backups of two devices. Another point and shoot comes in the mix and another DSLR comes in the mix.
Backing up all these devices to the hard drives that I use is risky if one drive fails. The file system I use is EXT4 a tried and tested stable file system.
I mitigated that problem by doubling the amount of hard drives. Now the amount of hard drives start to grow in such a manner that they do not fit in one case anymore. That means another machine had to be built to spread the amount of hard drives. There are significant costs, you need a motherboard a processor, memory, video output, a power supply, another case and you need to pay for electricity to power the new system.
You're starting to see the pattern and the risks involved when you don't want to lose data. When Android phones started to come in the mix it became really interesting.
You can use your Google account to store the images but you'll soon realize that even though they seem to give you a lot of space photographs fill it up exponentially fast.
Also you're storing it on somebody else's computer, who will abuse your data.
There has never been, there shall never be any cloud.
Soon you're faced with the fact that you need a network attached storage system NAS, where you can add drives and still use at the most two or three of those systems to manage your photographic data.
Without realizing it you've become a librarian, without the proper training and study which takes about 2 years!
If you have the computing & Database experience like me, it is easy to set up a plan to do proper archival storage.
The proper plan can include ZFS as a file system, which you will run natively in freeBSD or any other flavor. You can also choose HAMMER2 as your file system running in DragonFlyBSD, which has a very light footprint and a massive robustness built in, just like ZFS
For most people the financial factor will be a bottleneck, when you need to manage eight to twelve to even twenty-four camera devices, when you have stored photographs digitally for decades, starting from the beginning of DSLR camera bodies.
Any thoughts on #ZFS as the filesystem for a #Kubernetes node? My very unscientific initial sandbox impression is that it really squeezes the maximum performance out of the #Hetzner auction servers, which usually features a lot of slow rotating storage with a couple of #NVMe sticks thrown in to sweeten the deal. When the latter are used for caching, the throughput seems almost bearable. I'll do some more practical testing over the next weeks, but I'm quite optimistic. Maybe this will solve the disk saturation issues we're seeing, without breaking the budget.
So, yesterday, I listened to the latest episode of
@25admins, episode 256: Why ZFS.
If you're like me and have lots of interest in learning about ZFS, but still struggling to wrap your brain around parts of it, this is the episode for you. It's a gentle introduction to ZFS and why we should all use it. Some links to check out more info on ZFS are included in the show notes, as well.
Oh, and,
@joeress should no longer be considered the .5 of the group. He should get a bit of a promotion. 😀 Joe, you're killin' it!
What an excellent episode from yet another great
@latenightlinux podcast.
To celebrate the 256 milestone we devote the whole episode to explaining why we use ZFS. We explain about data safety, data retention, data portability, and ease of administration.
@splitbrain From what I’ve heard on
@25admins about #unraid I would not choose it no matter what, btrfs from own experience only on single disk setups. I even have #zfs on my laptop.
What's your opinion on Ubiquiti/Unifi gear?
I bought into the ecosystem while taking my networking cert classes back in 2017. They were much cheaper than Cisco gear for business-grade networking, and overall I've been happy with them. ...
A new BSDCan video has been posted: ZFS Direct IO Benchmarking Pitfalls by Mateusz Piotrowski ( youtu.be )
A new BSDCan video has been posted: ...