XPost: linux.debian.bugs.dist   
   From: pronoiac@gmail.com   
      
   Whoops, I hadn't intended to top-post... I'll do it correctly this time.   
      
   On Thu, Sep 25, 2025 at 12:57 PM james young  wrote:   
   >   
   > I'm not sure what timeline to expect for a response.   
   > Would a tarball of the outer image preserve everything needed for   
   diagnostics?   
   >   
   > -James   
   >   
   > On Tue, Sep 23, 2025 at 2:10 PM james young  wrote:   
   > >   
   > > I hit an issue with btrfs compression; I reported it to Debian, which   
   > > I was using, and they suggested that I take it upstream.   
   > >   
   > > Thanks, Salvatore. My apologies to everyone if I misunderstood.   
   > >   
   > > -James   
   > >   
   > > On Tue, Sep 23, 2025 at 1:50 PM Salvatore Bonaccorso    
   wrote:   
   > > >   
   > > > Control: tags -1 + moreinfo   
   > > >   
   > > > Hi James,   
   > > >   
   > > > On Tue, Sep 23, 2025 at 08:04:25PM +0200, James Young wrote:   
   > > > > Package: src:linux   
   > > > > Version: 6.1.129-1   
   > > > > Severity: normal   
   > > > > X-Debbugs-Cc: pronoiac@gmail.com   
   > > > >   
   > > > > Dear Maintainer,   
   > > > >   
   > > > >   
   > > > > * What led up to the situation?   
   > > > > We made empty files in a loop, in parallel, under CPU and I/O load.   
   > > > > We had an outer Btrfs image file with compression, which contained a   
   Btrfs image file, which contained billions of empty files.   
   > > > > We wrote around 100TB to the inner image file.   
   > > > > Around 60TB in, compression quietly shut off.   
   > > > > We ran out of space; both mounts presented i/o errors.   
   > > > >   
   > > > > * What exactly did you do (or not do) that was effective (or   
   ineffective)?   
   > > > >   * I unmounted the inner and outer images.   
   > > > >   I didn't take note of memory usage before this point.   
   > > > >   * dump debug info for the outer image - `btrfs inspect-internal   
   dump-tree --dfs ...`   
   > > > >   * We started a btrfsck. (twice, actually; breadth-first hit memory   
   limits, I think)   
   > > > > After that, I learned about `btrfs check`, but didn't interrupt the   
   btrfsck, due to Sunk Cost Fallacy.   
   > > > > The btrfsck is still running. It's of extremely dubious value now.   
   > > > > * check the kernel logs   
   > > > >   * I grepped for btrfs, the mount points, compress, and zstd. I   
   didn’t find a smoking gun in the right timeframe.   
   > > > >   
   > > > > not done yet:   
   > > > > * mount the outer image   
   > > > > * rebooted   
   > > > > * tried a newer kernel. we're currently on kernel 6.1.129; we could go   
   to newer 6.1 or 6.12 kernels   
   > > > > * redo live file system compression, with e.g. `btrfs filesystem   
   defrag -czstd`   
   > > > > * fstrim the outer image   
   > > > >   
   > > > > goals:   
   > > > > * work out what happened.   
   > > > > How can we help?   
   > > > > * help avoid it happening again, to others   
   > > > > * salvage what we can   
   > > > >   
   > > > > I've run `bugreport` as a non-privileged user. Let me know if root   
   access would give a fuller picture.   
   > > >   
   > > > I believe the best thing you could do here is to contact actually   
   > > > upstream people directly. get_maintainers and the MAINTAINERS file   
   > > > has:   
   > > >   
   > > > BTRFS FILE SYSTEM   
   > > > M:      Chris Mason    
   > > > M:      Josef Bacik    
   > > > M:      David Sterba    
   > > > L:      linux-btrfs@vger.kernel.org   
   > > > S:      Maintained   
   > > > W:      https://btrfs.readthedocs.io   
   > > > Q:      https://patchwork.kernel.org/project/linux-btrfs/list/   
   > > > C:      irc://irc.libera.chat/btrfs   
   > > > T:      git git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git   
   > > > F:      Documentation/filesystems/btrfs.rst   
   > > > F:      fs/btrfs/   
   > > > F:      include/linux/btrfs*   
   > > > F:      include/trace/events/btrfs.h   
   > > > F:      include/uapi/linux/btrfs*   
   > > >   
   > > > So I would suggest you to contact above maintainers including the   
   > > > list.   
   > > >   
   > > > Please keep this downstream bugreport as well in the recipients list.   
   > > >   
   > > > Regards,   
   > > > Salvatore   
      
   I made a tarball of the file system, then mounted and looked at the   
   file systems.   
   I attempted to recompress (with btrfs defrag) and fstrim, with little   
   success in freeing up space.   
      
   I started btrfs check with the progress option; within two hours, it   
   had gotten to “[2/7] checking extents, 82 items checked”.   
   I confused the extents with the compressed chunk length - 128KiB - so   
   that seemed woefully low on progress.   
   Over a week later, it’s still "82 items checked".   
   It’s still taking CPU (3% right now) and gigs of memory; it’s doing   
   something, though slowly.   
      
   So, a question:   
   * is this business as usual for a btrfs check?   
   * is this a clue about what happened?   
   * is this a symptom?   
      
   If this is a useful metric for file system robustness, is this   
   something I could / should experiment with to shorten?   
   * run `sync`   
   * periodically pause writes, to let the buffers empty   
      
      
   [continued in next message]   
      
   --- SoupGate-Win32 v1.05   
    * Origin: you cannot sedate... all the things you hate (1:229/2)