great start, btrfs is in read only mode, because no space left (no metadata space left). do i really have to reinstall?

@vbabka no way :)

@vbabka any rescue operation is blocked by the kernel remounting the fs as readonly because no space. so there is no way to recover from such a state?

@wagi @vbabka

I assume you already tried rebalacing?

It has been something like 10 years since I had such a problem. Back then a rebalance was able to fix such problems. Not sure if they require a r/w mount though.

@wagi @vbabka why? the author says it has no bugs and will not eat your data…

@kernellogger @vbabka I think I triggered this by booting into a snapshot and forgot to do 'snapper rollback, reboot'. This caused the metaspace to fillup. Anyway, all attempts to free same space fail, because 'free extents' will allocate new space. Unless there is an option to disable this. Also no chance to change the snappshot or release one

@ptesarik @vbabka Is he able to write bug free C code? I heard this is now a thing

@wagi @vbabka

And I assume you tried to follow guides like https://www.suse.com/support/kb/doc/?id=000018807 ?

@kernellogger @vbabka first step before reinstalling everthing full backup.

maybe I could fix the FS offdisk... e.g. copy the fs to a external space, free space, copy it back...

@kernellogger @vbabka yes, that's a funny guide. All btrfs rescue operation assume it's possible to mount the filesystem in read-write mode. Even adding additional space needs read-write fs. I wish there would be a userland tool which could do fs ops... maybe a hackweek project :)

@wagi @vbabka

🥴

Then good luck dealing with the situation!

@ptesarik @wagi @vbabka Yes, it will eat someone else's data. Now, imagine someone else is running bcachefs…

@kernellogger @vbabka Thanks, I'll recover eventually. This is just a learning experience.

I wont work anything today, that's for sure. the full disk backup takes 1h and then I can start with some destructive operations

@oleksandr @ptesarik @vbabka backups are for chickens!

@oleksandr @ptesarik @vbabka borg backup FTW!

@wagi @oleksandr @vbabka borg? ah, yes, all software written in the same language is bug-free…

@ptesarik @oleksandr @vbabka I am using https://www.borgbackup.org/ for a while. It's rsync on steroids and with a helper like borgmatic really really good to use. systemd timer triggers the backup. Take a snapshot and move it to the remote machine. Obviously encrypted.

@wagi @kernellogger @vbabka > This is just a learning experience

maybe you should have learnt that computers suck already sir

@lkundrak @kernellogger @vbabka it pays well

@wagi @kernellogger @vbabka i see.

*sigh*

is computer engineering basically onlyfans for those of us who don't look too good?

@ljs @kernellogger @lkundrak @vbabka obviously it exists: LonelyFans is a unique social platform tailored for individuals who embrace solitude and prefer their own company. It's a tongue-in-cheek, ...

@hyeyoo @kernellogger @lkundrak @ljs @vbabka lol https://web.archive.org/web/20231005155831/https://github.com/justinasr/FantilatorOS

@wagi @oleksandr @vbabka Yes, it actually looks good. Thank you for mentioning it.
And if that wasn't clear, I was merely teasing you, because you mentioned “bug-free C code” elsewhere in this thread.

@hyeyoo @kernellogger @wagi @lkundrak @ljs @vbabka The OS is intended only for project fans.

@ptesarik @oleksandr @vbabka haha, no worries. The C reference was from

https://mastodon.social/@sima@chaos.social/113429877869071458

@wagi @oleksandr @vbabka OK, but that one was “error free coding”. IIUC code is error-free when the compiler does not report any errors.

@ptesarik @oleksandr @vbabka this is how I work. if it compiles I ship it!

@wagi @ptesarik @oleksandr @vbabka "runtime error? you mean customer error?"

@lkundrak @ptesarik @oleksandr @vbabka

A scandinavian study showed that it's always a layer 8 problem.

@ptesarik @oleksandr @vbabka I suppose no one cares about it anyway. The CIP is happily pulling my tree

https://wiki.linuxfoundation.org/civilinfrastructureplatform/start

@wagi @kernellogger @vbabka sometimes it helps to user btrfs device add <usb stick > to give it some more spare space. Than it can be remounted rw, del some data and delete/remove the USB Stick from btrfs again

@manut @kernellogger @vbabka if there is no space left in the metadata zone this operation doesn't work. it's really sad, that if you hit the full metaspace situation the only way forward is to mkfs the partition. at least I didn't lose any data.

@wagi @kernellogger @vbabka damn! But thanks for explaining. I think I should add some monitoring for this on my machines that are heavily creating snapshots!