K. Ryabitsev 🍁
@monsieuricon
Moderator
Posts
2020Following
230Followers
2515This account is for Linux/Kernel/FOSS topics in general: #linux, #kernel, #foss, #git, #sysadmin, #infrastructure.
For my personal account, please follow @monsieuricon@castoranxieux.ca.
Montréal, Québec, Canada 🇨🇦🇺🇦
K. Ryabitsev 🍁
monsieuricon
✈️ ALA > DOH > YUL
I spent a great week in Almaty visiting my KZ cousins and meeting up with my parents. Now it's time to head back.
I spent a great week in Almaty visiting my KZ cousins and meeting up with my parents. Now it's time to head back.
0
0
2
K. Ryabitsev 🍁
monsieuricon
@iliazeus very good! I figured there aren't actually that many large canyons around the world, so someone is likely to figure it out. :)
0
0
1
K. Ryabitsev 🍁
monsieuricon
Hopping on an airplane to go home soon. Here are some horses from a walk earlier.
Any guesses where I have been?
Any guesses where I have been?
3
0
5
K. Ryabitsev 🍁
monsieuricon
@willywaschmaschine we're still using the 2017 release, but will switch to https://codeberg.org/mlmmj/mlmmj when there's a new release.
0
0
1
K. Ryabitsev 🍁
monsieuricon
Me: orders a delivery once while visiting Portugal in 2019
Routinely, for the next 4 years: "Atualização dos termos e condições"
Routinely, for the next 4 years: "Atualização dos termos e condições"
0
0
7
K. Ryabitsev 🍁
monsieuricon
@LWN reminds me of early days of net filtering when I was unable to download Microsoft Excel from my dorm computer because the url for "msexcel" had "sex" in it.
3
1
4
K. Ryabitsev 🍁
monsieuricon
@neil classically, a Unix socket that is 0660 group writable by the web process and readable by a monitoring process that invokes the privileged script. A more modern solution would be using dbus and systemd.
0
0
3
K. Ryabitsev 🍁
monsieuricon
re: Ask fedi: replicating large file collections over slow links
Show content
@mss Correct, where it says "replica-na" it should say "primary-na".
The question of temporary files is actually an important consideration. The content of primary-na is distro data that is copied to the system via rsync with --delay-updates, so everything is written into ~tmp~ dirs and then moved in-place at the end of a successful run. Theoretically, this should be handled correctly by fs-based replication.
The question of temporary files is actually an important consideration. The content of primary-na is distro data that is copied to the system via rsync with --delay-updates, so everything is written into ~tmp~ dirs and then moved in-place at the end of a successful run. Theoretically, this should be handled correctly by fs-based replication.
1
0
0
K. Ryabitsev 🍁
monsieuricon
@amonakov Yes, I find it intriguing for some of its concepts, but it also has a major downside of needing extra 50+ TB for storing the repository. Also, it is really written to solve a different problem -- backing up data as opposed to replicating it to multiple nodes efficiently.
0
0
0
K. Ryabitsev 🍁
monsieuricon
@asdil12 I'm aware of syncthing, but I'd need to see some evidence that it can scale up to 50TB and millions of files, properly recognize things like moves and hardlinks, etc. Unfortunately, I'm not in a position to easily experiment with it.
0
0
2
K. Ryabitsev 🍁
monsieuricon
Ask fedi: replicating large file collections over slow links
Show content
I have a server "primary-na" with 50TB of arbitrary content in /srv, mostly in millions of small files, many of them identical hardlinks. I have 3 other servers across the world (copy-na, copy-eu, copy-ap) where I want to have the exact replica of primary-na's /srv. These replicas may be occasionally unavailable for hours on end, or they may be occasionally slow or under high load. The content on them may also occasionally bitrot and must be identified and healed.
I've researched this multiple times over the last few years, but I've still not found a solution that would beat "just run rsync over it when something changes on replica-na." It's simple and effective, but obviously super inefficient and IO-heavy on both ends.
Any suggestions on how you would do it?
I've researched this multiple times over the last few years, but I've still not found a solution that would beat "just run rsync over it when something changes on replica-na." It's simple and effective, but obviously super inefficient and IO-heavy on both ends.
Any suggestions on how you would do it?
12
4
5