Extra PHDR section header in dbg file built via `--separate-debug-file`

[akarle]

First as always, a big thank you for all the hard work on mold. The performance gains have been incredibly impressive, especially with --separate-debug-file in a slower IO environment.

@mgulick and I noticed that gdb was significantly slower at symbol resolution with mold when we use --separate-debug-file, and while we're still looking into exactly what's going on in gdb, the thing that seems to stand out most is that there is this PHDR section header in the .dbg file that seems out of place (our best guess right now is that gdb chokes on it and starts searching the wrong section for symbols).

I've tried poking around the mold source code and see PHDR is a special "synthetic" chunk name that is intended to represent the Program Header, so it feels unintentional that it's being written into the section header (especially given its not there without --separate-debug-file), but I could be misunderstanding its purpose.

Can you confirm whether it's presence is expected? If it is, we can keep digging and hopefully get to a bug report for gdb.

---

Here's my minimal reproducer on Debian 12.12 using the latest mold commit (fcd5391) and gcc 12.2.0-14+deb12u1. Note that it seems somewhat sporadic for reasons I can't determine, so if it doesn't appear the first time you may need to relink a couple times.

$ cat foo.c
#include <stdio.h>
int foo(int f) {
  puts("hello world");
  return f + 1;
}

$ cat main.c
int foo(int);
int main() {
    return foo(0);
}

cc -g -O0 -c -fPIC foo.c
cc -fuse-ld=mold -Wl,--separate-debug-file -shared -o libfooBAD.so foo.o
cc -g -O0 main.c -o bad -L. -lfooBAD -Wl,-rpath -Wl,'$ORIGIN'

cc -fuse-ld=mold -shared -o libfooGOOD.so foo.o
objcopy --only-keep-debug libfooGOOD.so libfooGOOD.so.dbg
strip libfooGOOD.so
objcopy --add-gnu-debuglink=libfooGOOD.so.dbg libfooGOOD.so
cc -g -O0 main.c -o good -L. -lfooGOOD -Wl,-rpath -Wl,'$ORIGIN'

readelf -SW libfooBAD.so.dbg | grep PHDR
  [ 1] PHDR              NOBITS          0000000000000040 000040 000230 00   A  0   0  8

readelf -SW libfooGOOD.so.dbg | grep PHDR
[empty]

And here's the full symbol table

$ readelf -SW libfooBAD.so.dbg
There are 38 section headers, starting at offset 0x4a8:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] PHDR              NOBITS          0000000000000040 000040 000230 00   A  0   0  8
  [ 2] .note.gnu.build-id NOTE            0000000000000270 000270 000024 00   A  0   0  4
  [ 3] .gnu.hash         NOBITS          0000000000000298 000294 000020 00   A  3   0  8
  [ 4] .dynsym           NOBITS          00000000000002b8 000294 0000a8 18   A  4   1  8
  [ 5] .dynstr           NOBITS          0000000000000360 000294 000076 00   A  0   0  1
  [ 6] .gnu.version      NOBITS          00000000000003d6 000294 00000e 02   A  3   0  2
  [ 7] .gnu.version_r    NOBITS          00000000000003e4 000294 000020 00   A  4   1  4
  [ 8] .rela.dyn         NOBITS          0000000000000408 000294 0000a8 18   A  3   0  8
  [ 9] .rela.plt         NOBITS          00000000000004b0 000294 000018 18   A  3  23  8
  [10] .eh_frame         NOBITS          00000000000004c8 000294 00003c 00   A  0   0  8
  [11] .eh_frame_hdr     NOBITS          0000000000000504 000294 000014 00   A  0   0  4
  [12] .rodata           NOBITS          0000000000000518 000294 000031 00   A  0   0  8
  [13] .fini             NOBITS          000000000000154c 000294 000009 00  AX  0   0  4
  [14] .init             NOBITS          0000000000001558 000294 000017 00  AX  0   0  4
  [15] .plt              NOBITS          0000000000001570 000294 000030 00  AX  0   0 16
  [16] .plt.got          NOBITS          00000000000015a0 000294 000008 00  AX  0   0 16
  [17] .text             NOBITS          00000000000015b0 000294 0000ee 00  AX  0   0 16
  [18] .dynamic          NOBITS          00000000000026a0 000294 0001c0 10  WA  4   0  8
  [19] .fini_array       NOBITS          0000000000002860 000294 000008 00  WA  0   0  8
  [20] .init_array       NOBITS          0000000000002868 000294 000008 00  WA  0   0  8
  [21] .got              NOBITS          0000000000002870 000294 000028 00  WA  0   0  8
  [22] .relro_padding    NOBITS          0000000000002898 000294 000768 00  WA  0   0  1
  [23] .data             NOBITS          0000000000003000 000294 000008 00  WA  0   0  8
  [24] .got.plt          NOBITS          0000000000003008 000294 000020 00  WA  0   0  8
  [25] .tm_clone_table   NOBITS          0000000000003028 000294 000000 00  WA  0   0  8
  [26] .bss              NOBITS          0000000000003028 000294 000001 00  WA  0   0  1
  [27] .comment          NOBITS          0000000000000000 000294 000077 01  MS  0   0  1
  [28] .gnu_debuglink    NOBITS          0000000000000000 00030c 000018 00      0   0  4
  [29] .shstrtab         STRTAB          0000000000000000 000324 00017e 00      0   0  1
  [30] .debug_abbrev     PROGBITS        0000000000000000 000e28 00009a 00      0   0  1
  [31] .debug_aranges    PROGBITS        0000000000000000 000ec2 000030 00      0   0  1
  [32] .debug_info       PROGBITS        0000000000000000 000ef2 0000cc 00      0   0  1
  [33] .debug_line       PROGBITS        0000000000000000 000fbe 00005f 00      0   0  1
  [34] .debug_line_str   PROGBITS        0000000000000000 00101d 00004f 01  MS  0   0  1
  [35] .debug_str        PROGBITS        0000000000000000 00106c 0000c1 01  MS  0   0  1
  [36] .strtab           STRTAB          0000000000000000 00112d 0002bd 00      0   0  1
  [37] .symtab           SYMTAB          0000000000000000 0013f0 0007f8 18     36  79  8
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  D (mbind), l (large), p (processor specific)

the program header on the debug info is also nulled out:

$ readelf -a libfooBAD.so.dbg

...
Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x0
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x0
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x0
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x0
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x0
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x0
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x0
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x0
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x0

[akarle]

I think the solution may be as simple as this:

diff --git a/src/passes.cc b/src/passes.cc
index 6eaecf3c..fee65756 100644
--- a/src/passes.cc
+++ b/src/passes.cc
@@ -3398,7 +3398,7 @@ void write_separate_debug_file(Context<E> &ctx) {
   // real sections into dummy sections here.
   for (i64 i = 0; i < ctx.chunks.size(); i++) {
     Chunk<E> *chunk = ctx.chunks[i];
-    if (chunk != ctx.ehdr && chunk != ctx.shdr && chunk != ctx.shstrtab &&
+    if (chunk != ctx.ehdr && chunk != ctx.shdr && chunk != ctx.phdr && chunk != ctx.shstrtab &&
         chunk->shdr.sh_type != SHT_NOTE) {
       Chunk<E> *sec = new OutputSection<E>(chunk->name, SHT_NULL);
       sec->shdr = chunk->shdr;

But I don't have enough context to say whether it's right for sure.

EDIT: no, this is causing other weirdness... I don't think it's quite right, but I also am hoping its a similarly sized fix elsewhere. More specifically, with this change, the program header is non-null and the PHDR section header is gone, but now I'm unable to set a breakpoint.

[akarle]

Ah I think I figured out why gdb is looking in the wrong section--the symbol foo is pointing at section 16 which should be .text but is .plt.got because of the added PHDR:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
...
  [14] .init             NOBITS          000000000000152c 000294 000017 00  AX  0   0  4
  [15] .plt              NOBITS          0000000000001550 000294 000030 00  AX  0   0 16
  [16] .plt.got          NOBITS          0000000000001580 000294 000008 00  AX  0   0 16
  [17] .text             NOBITS          0000000000001590 000294 0000db 00  AX  0   0 16
  [18] .dynamic          NOBITS          0000000000002670 000294 0001c0 10  WA  4   0  8
...


Symbol table '.symtab' contains 85 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 0000000000000040     0 SECTION LOCAL  DEFAULT    1 PHDR
...
    82: 0000000000001649    34 FUNC    GLOBAL DEFAULT   16 foo

so it's looking less like gdb is doing the wrong thing due to the inability to parse the PHDR section header and more like the data itself in the dbg is telling gdb to go look in the wrong place.

Now, with my patch applied, the section headers look good, and foo is in the right section:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
...
  [14] .plt              NOBITS          00000000000014e0 000224 000030 00  AX  0   0 16
  [15] .plt.got          NOBITS          0000000000001510 000224 000008 00  AX  0   0 16
  [16] .text             NOBITS          0000000000001520 000224 0000db 00  AX  0   0 16

...

    81: 0000000000001649    34 FUNC    GLOBAL DEFAULT   16 foo

but now the offsets look wrong--.text is from 1520->1250+db and foo is in 1649, which is out of bounds (explaining I think the breakpoint failing).

in my non-separate-debug-file-case (libfooGOOD.so.dbg), it looks fine:

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
...
  [17] .plt.got          NOBITS          00000000000015b0 000558 000008 00  AX  0   0 16
  [18] .text             NOBITS          00000000000015c0 000558 0000db 00  AX  0   0 16
  [19] .dynamic          NOBITS          00000000000026a0 0006a0 0001c0 10  WA  6   0  8
  [20] .fini_array       NOBITS          0000000000002860 0006a0 000008 00  WA  0   0  8
...
    51: 0000000000001679    34 FUNC    GLOBAL DEFAULT   18 foo

I'm still unclear why the patch is changing the section address location...

[rui314]

It looks like a bug in mold. PHDR chunk should never appear in the section header. Let me reproduce the issue locally to debug it.

[rui314]

Does the above change work for you guys?

[akarle]

Thanks for the quick reply! I tried this change and I think it suffers a different type of corruption (I was trying to convey this in my second and third message but admittedly it's verbose/obscure).

Using the reproducer from the main description where I have libfooBAD.so and main.c, I can see that PHDR is no longer in the section headers (which is good), but now the memory locations look wrong, which prevents GDB from setting a breakpoint in foo:

$ gdb bad
(gdb) start
(gdb) b foo
(gdb) c
Breakpoint 1.1, 0x00007ffff7fbf64d in foo () from /tmp/libfooBAD.so
-- doesn't resolve to the source file

My non-expert understanding is because the .dbg file has the .text section being mapped into the wrong area of memory. Look how the .so maps .text into 1590 with foo at 1649, but the .dbg maps .text into 1520 with foo still at 1649. Given that .text is only of size db, 1649 is out of range of 1520+db

$ readelf -Wa libfooBAD.so | egrep '(foo|] \.text)'
  [16] .text             PROGBITS        0000000000001590 000590 0000db 00  AX  0   0 16
     6: 0000000000001649    34 FUNC    GLOBAL DEFAULT   16 foo

$ readelf -Wa libfooBAD.so.dbg | egrep '(foo|] \.text)'
  [16] .text             NOBITS          0000000000001520 000224 0000db 00  AX  0   0 16
    49: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS foo.c
    81: 0000000000001649    34 FUNC    GLOBAL DEFAULT   16 foo

Based on how mold behaves when not using --separate-debug-file but instead using objcopy+strip to generate the .dbg file (libfooGOOD), I'd expect .text to retain the same start address:

$ readelf -Wa libfooGOOD.so | egrep '(foo|] \.text)'
  [16] .text             PROGBITS        0000000000001590 000590 0000db 00  AX  0   0 16
     6: 0000000000001649    34 FUNC    GLOBAL DEFAULT   16 foo

$ readelf -Wa libfooGOOD.so.dbg | egrep '(foo|] \.text)'
  [16] .text             NOBITS          0000000000001590 000520 0000db 00  AX  0   0 16
    19: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS foo.c
    51: 0000000000001649    34 FUNC    GLOBAL DEFAULT   16 foo

Hopefully that makes sense! I'm still learning about ELF file layout, so some of this is me trying to map what I'm seeing in gdb to what looks different between the binaries.

Thank you again for the quick help--it's greatly appreciated!

[rui314]

How about the above one?

[akarle]

Thanks for the followup! We seem closer, but something is still off. I wish I could describe it better and/or pin down a reproduction, but it's very weirdly dependent on things as trivial as the compilation directory.

We're seeing the separate dbg file now have some corruption. In its worst form, objcopy doesn't even recognize the file format:

objcopy: myprod.so.dbg symbol number 6038 references nonexistent SHT_SYMTAB_SHNDX section
objcopy: myprod.so.dbg: file format not recognized

I can't share myprod.so.dbg, but we've corrupted our minimal example too:

$ objcopy libfooBAD.so.dbg /tmp/xxx
objcopy: libfooBAD.so.dbg: invalid string offset 10504 >= 830 for section `.strtab'
objcopy: libfooBAD.so.dbg: invalid string offset 14600 >= 830 for section `.strtab'
objcopy: libfooBAD.so.dbg: invalid string offset 14608 >= 830 for section `.strtab'
objcopy: libfooBAD.so.dbg: invalid string offset 14640 >= 830 for section `.strtab'
objcopy: libfooBAD.so.dbg: invalid string offset 14640 >= 830 for section `.strtab'

gdb outputs these messages too. The thing that sticks out most is that in the .symtab, we're seeing these extra, nameless, undefined symbols that are corrupt:

ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          64 (bytes into file)
  Start of section headers:          1040 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         8
  Size of section headers:           64 (bytes)
  Number of section headers:         37
  Section header string table index: 28

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .note.gnu.build-id NOTE            0000000000000270 000270 000024 00   A  0   0  4
  [ 2] .gnu.hash         NOBITS          0000000000000298 000294 000020 00   A  3   0  8
  [ 3] .dynsym           NOBITS          00000000000002b8 000294 0000a8 18   A  4   1  8
  [ 4] .dynstr           NOBITS          0000000000000360 000294 000074 00   A  0   0  1
  [ 5] .gnu.version      NOBITS          00000000000003d4 000294 00000e 02   A  3   0  2
  [ 6] .gnu.version_r    NOBITS          00000000000003e4 000294 000020 00   A  4   1  4
  [ 7] .rela.dyn         NOBITS          0000000000000408 000294 0000a8 18   A  3   0  8
  [ 8] .rela.plt         NOBITS          00000000000004b0 000294 000018 18   A  3  23  8
  [ 9] .eh_frame         NOBITS          00000000000004c8 000294 00003c 00   A  0   0  8
  [10] .eh_frame_hdr     NOBITS          0000000000000504 000294 000014 00   A  0   0  4
  [11] .rodata           NOBITS          0000000000000518 000294 00000c 00   A  0   0  1
  [12] .fini             NOBITS          0000000000001524 000294 000009 00  AX  0   0  4
  [13] .init             NOBITS          0000000000001530 000294 000017 00  AX  0   0  4
  [14] .plt              NOBITS          0000000000001550 000294 000030 00  AX  0   0 16
  [15] .plt.got          NOBITS          0000000000001580 000294 000008 00  AX  0   0 16
  [16] .text             NOBITS          0000000000001590 000294 0000db 00  AX  0   0 16
  [17] .dynamic          NOBITS          0000000000002670 000294 0001c0 10  WA  4   0  8
  [18] .fini_array       NOBITS          0000000000002830 000294 000008 00  WA  0   0  8
  [19] .init_array       NOBITS          0000000000002838 000294 000008 00  WA  0   0  8
  [20] .got              NOBITS          0000000000002840 000294 000028 00  WA  0   0  8
  [21] .relro_padding    NOBITS          0000000000002868 000294 000798 00  WA  0   0  1
  [22] .data             NOBITS          0000000000003868 000294 000008 00  WA  0   0  8
  [23] .got.plt          NOBITS          0000000000003870 000294 000020 00  WA  0   0  8
  [24] .tm_clone_table   NOBITS          0000000000003890 000294 000000 00  WA  0   0  8
  [25] .bss              NOBITS          0000000000003890 000294 000001 00  WA  0   0  1
  [26] .comment          NOBITS          0000000000000000 000294 000077 01  MS  0   0  1
  [27] .gnu_debuglink    NOBITS          0000000000000000 000294 000018 00      0   0  4
  [28] .shstrtab         STRTAB          0000000000000000 000294 000179 00      0   0  1
  [29] .debug_abbrev     PROGBITS        0000000000000000 000d50 000084 00      0   0  1
  [30] .debug_aranges    PROGBITS        0000000000000000 000dd4 000030 00      0   0  1
  [31] .debug_info       PROGBITS        0000000000000000 000e04 0000bc 00      0   0  1
  [32] .debug_line       PROGBITS        0000000000000000 000ec0 00005b 00      0   0  1
  [33] .debug_line_str   PROGBITS        0000000000000000 000f1b 00004a 01  MS  0   0  1
  [34] .debug_str        PROGBITS        0000000000000000 000f65 0000bf 01  MS  0   0  1
  [35] .strtab           STRTAB          0000000000000000 001024 00033e 00      0   0  1
  [36] .symtab           SYMTAB          0000000000000000 001368 000870 18     35  84  8
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  D (mbind), l (large), p (processor specific)

There are no section groups in this file.

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  PHDR           0x000040 0x0000000000000040 0x0000000000000040 0x000230 0x000230 R   0x8
  NOTE           0x000270 0x0000000000000270 0x0000000000000270 0x000024 0x000024 R   0x4
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x000294 0x000524 R   0x1000
  LOAD           0x000524 0x0000000000001524 0x0000000000001524 0x000000 0x000147 R E 0x1000
  LOAD           0x000670 0x0000000000002670 0x0000000000002670 0x000000 0x001221 RW  0x1000
  DYNAMIC        0x000670 0x0000000000002670 0x0000000000002670 0x0001c0 0x0001c0 RW  0x8
  GNU_EH_FRAME   0x000504 0x0000000000000504 0x0000000000000504 0x000014 0x000014 R   0x4
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x1

 Section to Segment mapping:
  Segment Sections...
   00     
   01     .note.gnu.build-id .comment .gnu_debuglink 
   02     .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .eh_frame .eh_frame_hdr .rodata 
   03     .fini .init .plt .plt.got .text 
   04     .dynamic .fini_array .init_array .got .relro_padding .data .got.plt .tm_clone_table .bss 
   05     .dynamic 
   06     .eh_frame_hdr 
   07     

There is no dynamic section in this file.

There are no relocations in this file.
No processor specific unwind information to decode

Symbol table '.symtab' contains 90 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000270     0 SECTION LOCAL  DEFAULT    1 .note.gnu.build-id
     2: 0000000000000298     0 SECTION LOCAL  DEFAULT    2 .gnu.hash
     3: 00000000000002b8     0 SECTION LOCAL  DEFAULT    3 .dynsym
     4: 0000000000000360     0 SECTION LOCAL  DEFAULT    4 .dynstr
     5: 00000000000003d4     0 SECTION LOCAL  DEFAULT    5 .gnu.version
     6: 00000000000003e4     0 SECTION LOCAL  DEFAULT    6 .gnu.version_r
     7: 0000000000000408     0 SECTION LOCAL  DEFAULT    7 .rela.dyn
     8: 00000000000004b0     0 SECTION LOCAL  DEFAULT    8 .rela.plt
     9: 00000000000004c8     0 SECTION LOCAL  DEFAULT    9 .eh_frame
    10: 0000000000000504     0 SECTION LOCAL  DEFAULT   10 .eh_frame_hdr
    11: 0000000000000518     0 SECTION LOCAL  DEFAULT   11 .rodata
    12: 0000000000001524     0 SECTION LOCAL  DEFAULT   12 .fini
    13: 0000000000001530     0 SECTION LOCAL  DEFAULT   13 .init
    14: 0000000000001550     0 SECTION LOCAL  DEFAULT   14 .plt
    15: 0000000000001580     0 SECTION LOCAL  DEFAULT   15 .plt.got
    16: 0000000000001590     0 SECTION LOCAL  DEFAULT   16 .text
    17: 0000000000002670     0 SECTION LOCAL  DEFAULT   17 .dynamic
    18: 0000000000002830     0 SECTION LOCAL  DEFAULT   18 .fini_array
    19: 0000000000002838     0 SECTION LOCAL  DEFAULT   19 .init_array
    20: 0000000000002840     0 SECTION LOCAL  DEFAULT   20 .got
    21: 0000000000002868     0 SECTION LOCAL  DEFAULT   21 .relro_padding
    22: 0000000000003868     0 SECTION LOCAL  DEFAULT   22 .data
    23: 0000000000003870     0 SECTION LOCAL  DEFAULT   23 .got.plt
    24: 0000000000003890     0 SECTION LOCAL  DEFAULT   24 .tm_clone_table
    25: 0000000000003890     0 SECTION LOCAL  DEFAULT   25 .bss
    26: 0000000000000000     0 SECTION LOCAL  DEFAULT   26 .comment
    27: 0000000000000000     0 SECTION LOCAL  DEFAULT   27 .gnu_debuglink
    28: 0000000000000000     0 SECTION LOCAL  DEFAULT   28 .shstrtab
    29: 0000000000000000     0 SECTION LOCAL  DEFAULT   29 .debug_abbrev
    30: 0000000000000000     0 SECTION LOCAL  DEFAULT   30 .debug_aranges
    31: 0000000000000000     0 SECTION LOCAL  DEFAULT   31 .debug_info
    32: 0000000000000000     0 SECTION LOCAL  DEFAULT   32 .debug_line
    33: 0000000000000000     0 SECTION LOCAL  DEFAULT   33 .debug_line_str
    34: 0000000000000000     0 SECTION LOCAL  DEFAULT   34 .debug_str
    35: 0000000000000000     0 SECTION LOCAL  DEFAULT   35 .strtab
    36: 0000000000000000     0 SECTION LOCAL  DEFAULT   36 .symtab
    37: 0000000000000000 0x18000300000000 NOTYPE  LOCAL  DEFAULT  UND <corrupt>
    38: 0000000000000000 0x19000300000000 NOTYPE  LOCAL  DEFAULT  UND <corrupt>
    39: 0000000000000000 0x1a000300000000 NOTYPE  LOCAL  DEFAULT  UND <corrupt>
    40: 0000000000000000 0x1b000300000000 NOTYPE  LOCAL  DEFAULT  UND <corrupt>
    41: 0000000000000000 0x1c000300000000 NOTYPE  LOCAL  DEFAULT  UND <corrupt>
    42: 0000000000000000 0x1d000300000000 NOTYPE  LOCAL  DEFAULT  UND 
    43: 0000000000001530     0 FUNC    LOCAL  HIDDEN    13 _init
    44: 0000000000001524     0 FUNC    LOCAL  HIDDEN    12 _fini
    45: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS crtstuff.c
    46: 0000000000003890     0 OBJECT  LOCAL  DEFAULT   24 __TMC_LIST__
    47: 0000000000001590     0 FUNC    LOCAL  DEFAULT   16 deregister_tm_clones
    48: 00000000000015c0     0 FUNC    LOCAL  DEFAULT   16 register_tm_clones
    49: 0000000000001600     0 FUNC    LOCAL  DEFAULT   16 __do_global_dtors_aux
    50: 0000000000003890     1 OBJECT  LOCAL  DEFAULT   25 completed.0
    51: 0000000000002830     0 OBJECT  LOCAL  DEFAULT   18 __do_global_dtors_aux_fini_array_entry
    52: 0000000000001640     0 FUNC    LOCAL  DEFAULT   16 frame_dummy
    53: 0000000000002838     0 OBJECT  LOCAL  DEFAULT   19 __frame_dummy_init_array_entry
    54: 0000000000003868     0 OBJECT  LOCAL  HIDDEN    22 __dso_handle
    55: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS foo.c
    56: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS crtstuff.c
    57: 0000000000003890     0 OBJECT  LOCAL  HIDDEN    24 __TMC_END__
    58: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT    1 __ehdr_start
    59: 0000000000002838     0 NOTYPE  LOCAL  DEFAULT   19 __init_array_start
    60: 0000000000002840     0 NOTYPE  LOCAL  DEFAULT   19 __init_array_end
    61: 0000000000002830     0 NOTYPE  LOCAL  DEFAULT   18 __fini_array_start
    62: 0000000000002838     0 NOTYPE  LOCAL  DEFAULT   18 __fini_array_end
    63: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   36 __preinit_array_start
    64: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   36 __preinit_array_end
    65: 0000000000002670     0 NOTYPE  LOCAL  DEFAULT   17 _DYNAMIC
    66: 0000000000003870     0 NOTYPE  LOCAL  DEFAULT   23 _GLOBAL_OFFSET_TABLE_
    67: 0000000000001550     0 NOTYPE  LOCAL  DEFAULT   14 _PROCEDURE_LINKAGE_TABLE_
    68: 0000000000003890     0 NOTYPE  LOCAL  DEFAULT   25 __bss_start
    69: 0000000000003891     0 NOTYPE  LOCAL  DEFAULT   25 _end
    70: 000000000000166b     0 NOTYPE  LOCAL  DEFAULT   16 _etext
    71: 0000000000003890     0 NOTYPE  LOCAL  DEFAULT   24 _edata
    72: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT    1 __executable_start
    73: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  ABS __rela_iplt_start
    74: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  ABS __rela_iplt_end
    75: 0000000000000504     0 NOTYPE  LOCAL  DEFAULT   10 __GNU_EH_FRAME_HDR
    76: 0000000000003891     0 NOTYPE  LOCAL  DEFAULT   25 end
    77: 000000000000166b     0 NOTYPE  LOCAL  DEFAULT   16 etext
    78: 0000000000003890     0 NOTYPE  LOCAL  DEFAULT   24 edata
    79: 0000000000000000     0 TLS     LOCAL  DEFAULT    1 _TLS_MODULE_BASE_
    80: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   36 __start_EHDR
    81: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   36 __stop_EHDR
    82: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   36 __start_PHDR
    83: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   36 __stop_PHDR
    84: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
    85: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_deregisterTMCloneTable
    86: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _ITM_registerTMCloneTable
    87: 0000000000001649    34 FUNC    GLOBAL DEFAULT   16 foo
    88: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND puts
    89: 0000000000000000     0 FUNC    WEAK   DEFAULT  UND __cxa_finalize

No version information found in this file.

Displaying notes found in: .note.gnu.build-id
  Owner                Data size 	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build ID: 2811c39989fd0fee8e697ee9f0a19681925998c2

certain compiler / compile path combinations seem to push this corruption elsewhere (I've even seen it build without any corruption), but it doesn't really have much rhyme or reason to it.

It really seems like it's writing data that shouldn't be in the symtab to the symtab (and corrupting it). But at least the section header now looks correct (.text is now at 1590) :)

GitHub doesn't allow attaching .dbg files, so I put a txt at the end--it's the dbg info though! I'm hoping that you can reproduce some wonkiness/corruption on your end though.

libfooBADsodbg.txt

[akarle]

Here's the start of the corrupt production dbg data, in case it helps. The first 160k symbols seem to all have some form of:

bad section index[NNNN] <corrupt>

OR

<corrupt>

OR

binary garbage

and then after that it looks fine at a glance... the sizes being massive also feels wrong.

ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          64 (bytes into file)
  Start of section headers:          1288 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         10
  Size of section headers:           64 (bytes)
  Number of section headers:         44
  Section header string table index: 33

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .note.gnu.property NOTE            00000000000002e0 0002e0 000030 00   A  0   0  8
  [ 2] .note.gnu.build-id NOTE            0000000000000310 000310 000024 00   A  0   0  4
  [ 3] .hash             NOBITS          0000000000000334 000334 18d598 04   A  5   0  4
  [ 4] .gnu.hash         NOBITS          000000000018d8d0 000334 15bdd4 00   A  5   0  8
  [ 5] .dynsym           NOBITS          00000000002e96a8 000334 4a80b0 18   A  6   1  8
  [ 6] .dynstr           NOBITS          0000000000791758 000334 1c500f2 00   A  0   0  1
  [ 7] .gnu.version      NOBITS          00000000023e184a 000334 063564 02   A  5   0  2
  [ 8] .gnu.version_r    NOBITS          0000000002444db0 000334 000260 00   A  6   7  4
  [ 9] .rela.dyn         NOBITS          0000000002445010 000334 0aba50 18   A  5   0  8
  [10] .rela.plt         NOBITS          00000000024f0a60 000334 392850 18   A  5  28  8
  [11] .eh_frame         NOBITS          00000000028832b0 000334 5f33a4 00   A  0   0  8
  [12] .eh_frame_hdr     NOBITS          0000000002e76654 000334 16f1cc 00   A  0   0  4
  [13] .gcc_except_table NOBITS          0000000002fe5820 000334 084585 00   A  0   0  4
  [14] .rodata           NOBITS          0000000003069dc0 000334 501e4c 00   A  0   0 32
  [15] .fini             NOBITS          000000000356cc0c 000334 000009 00  AX  0   0  4
  [16] .init             NOBITS          000000000356cc18 000334 000017 00  AX  0   0  4
  [17] .plt              NOBITS          000000000356cc30 000334 261b00 00  AX  0   0 16
  [18] .plt.got          NOBITS          00000000037ce730 000334 000a30 00  AX  0   0 16
  [19] .text             NOBITS          00000000037cf160 000334 102e29b 00  AX  0   0 16
  [20] .tbss             NOBITS          00000000047fe400 000334 000008 00 WAT  0   0  8
  [21] .data.rel.ro      NOBITS          00000000047fe400 000334 033328 00  WA  0   0 32
  [22] .dynamic          NOBITS          0000000004831728 000334 000520 10  WA  6   0  8
  [23] .fini_array       NOBITS          0000000004831c48 000334 000008 00  WA  0   0  8
  [24] .init_array       NOBITS          0000000004831c50 000334 000b90 00  WA  0   0  8
  [25] .got              NOBITS          00000000048327e0 000334 007f60 00  WA  0   0  8
  [26] .relro_padding    NOBITS          000000000483a740 000334 0008c0 00  WA  0   0  1
  [27] .data             NOBITS          000000000483b740 000334 003bf8 00  WA  0   0 32
  [28] .got.plt          NOBITS          000000000483f338 000334 130d88 00  WA  0   0  8
  [29] .tm_clone_table   NOBITS          00000000049700c0 000334 000000 00  WA  0   0  8
  [30] .bss              NOBITS          00000000049700c0 000334 013c20 00  WA  0   0 32
  [31] .comment          NOBITS          0000000000000000 000334 00007a 01  MS  0   0  1
  [32] .gnu_debuglink    NOBITS          0000000000000000 000334 000014 00      0   0  4
  [33] .shstrtab         STRTAB          0000000000000000 000334 0001d2 00      0   0  1
  [34] .debug_abbrev     PROGBITS        0000000000000000 001008 53c09d 00      0   0  1
  [35] .debug_aranges    PROGBITS        0000000000000000 53d0a5 75b870 00      0   0  1
  [36] .debug_info       PROGBITS        0000000000000000 c98915 1ffe180d 00      0   0  1
  [37] .debug_line       PROGBITS        0000000000000000 20c7a122 1b3f331 00      0   0  1
  [38] .debug_line_str   PROGBITS        0000000000000000 227b9453 022af5 01  MS  0   0  1
  [39] .debug_rnglists   PROGBITS        0000000000000000 227dbf48 4c9412 00      0   0  1
  [40] .debug_str        PROGBITS        0000000000000000 22ca535a 9376ade 01  MS  0   0  1
  [41] .strtab           STRTAB          0000000000000000 2c01be38 4c7d744 00      0   0  1
  [42] .symtab           SYMTAB          0000000000000000 30c99580 d5d368 18     41 380446  8
  [43] .gdb_index        PROGBITS        0000000000000000 319f68e8 928f035 00      0   0  4
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  D (mbind), l (large), p (processor specific)

There are no section groups in this file.

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  PHDR           0x000040 0x0000000000000040 0x0000000000000040 0x0002a0 0x0002a0 R   0x8
  NOTE           0x0002e0 0x00000000000002e0 0x00000000000002e0 0x000054 0x000054 R   0x8
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x000334 0x356bc0c R   0x1000
  LOAD           0x000c0c 0x000000000356cc0c 0x000000000356cc0c 0x000000 0x12907ef R E 0x1000
  LOAD           0x000400 0x00000000047fe400 0x00000000047fe400 0x000000 0x1858e0 RW  0x1000
  TLS            0x000400 0x00000000047fe400 0x00000000047fe400 0x000000 0x000008 R   0x8
  DYNAMIC        0x482f728 0x0000000004831728 0x0000000004831728 0x000520 0x000520 RW  0x8
  GNU_EH_FRAME   0x2e76654 0x0000000002e76654 0x0000000002e76654 0x16f1cc 0x16f1cc R   0x4
  GNU_PROPERTY   0x0002e0 0x00000000000002e0 0x00000000000002e0 0x000030 0x000030 R   0x8
  GNU_STACK      0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW  0x1

 Section to Segment mapping:
  Segment Sections...
   00     
   01     .note.gnu.property .note.gnu.build-id .comment .gnu_debuglink 
   02     .note.gnu.property .note.gnu.build-id .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .eh_frame .eh_frame_hdr .gcc_except_table .rodata 
   03     .fini .init .plt .plt.got .text 
   04     .data.rel.ro .dynamic .fini_array .init_array .got .relro_padding .data .got.plt .tm_clone_table .bss 
   05     .tbss 
   06     .dynamic 
   07     .eh_frame_hdr 
   08     .note.gnu.property .comment .gnu_debuglink 
   09     

There is no dynamic section in this file.

There are no relocations in this file.
No processor specific unwind information to decode

Symbol table '.symtab' contains 583887 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 00000000000002e0     0 SECTION LOCAL  DEFAULT    1 .note.gnu.property
     2: 0000000000000310     0 SECTION LOCAL  DEFAULT    2 .note.gnu.build-id
     3: 0000000000000334     0 SECTION LOCAL  DEFAULT    3 .hash
     4: 000000000018d8d0     0 SECTION LOCAL  DEFAULT    4 .gnu.hash
     5: 00000000002e96a8     0 SECTION LOCAL  DEFAULT    5 .dynsym
     6: 0000000000791758     0 SECTION LOCAL  DEFAULT    6 .dynstr
     7: 00000000023e184a     0 SECTION LOCAL  DEFAULT    7 .gnu.version
     8: 0000000002444db0     0 SECTION LOCAL  DEFAULT    8 .gnu.version_r
     9: 0000000002445010     0 SECTION LOCAL  DEFAULT    9 .rela.dyn
    10: 00000000024f0a60     0 SECTION LOCAL  DEFAULT   10 .rela.plt
    11: 00000000028832b0     0 SECTION LOCAL  DEFAULT   11 .eh_frame
    12: 0000000002e76654     0 SECTION LOCAL  DEFAULT   12 .eh_frame_hdr
    13: 0000000002fe5820     0 SECTION LOCAL  DEFAULT   13 .gcc_except_table
    14: 0000000003069dc0     0 SECTION LOCAL  DEFAULT   14 .rodata
    15: 000000000356cc0c     0 SECTION LOCAL  DEFAULT   15 .fini
    16: 000000000356cc18     0 SECTION LOCAL  DEFAULT   16 .init
    17: 000000000356cc30     0 SECTION LOCAL  DEFAULT   17 .plt
    18: 00000000037ce730     0 SECTION LOCAL  DEFAULT   18 .plt.got
    19: 00000000037cf160     0 SECTION LOCAL  DEFAULT   19 .text
    20: 00000000047fe400     0 SECTION LOCAL  DEFAULT   20 .tbss
    21: 00000000047fe400     0 SECTION LOCAL  DEFAULT   21 .data.rel.ro
    22: 0000000004831728     0 SECTION LOCAL  DEFAULT   22 .dynamic
    23: 0000000004831c48     0 SECTION LOCAL  DEFAULT   23 .fini_array
    24: 0000000004831c50     0 SECTION LOCAL  DEFAULT   24 .init_array
    25: 00000000048327e0     0 SECTION LOCAL  DEFAULT   25 .got
    26: 000000000483a740     0 SECTION LOCAL  DEFAULT   26 .relro_padding
    27: 000000000483b740     0 SECTION LOCAL  DEFAULT   27 .data
    28: 000000000483f338     0 SECTION LOCAL  DEFAULT   28 .got.plt
    29: 00000000049700c0     0 SECTION LOCAL  DEFAULT   29 .tm_clone_table
    30: 00000000049700c0     0 SECTION LOCAL  DEFAULT   30 .bss
    31: 0000000000000000     0 SECTION LOCAL  DEFAULT   31 .comment
    32: 0000000000000000     0 SECTION LOCAL  DEFAULT   32 .gnu_debuglink
    33: 0000000000000000     0 SECTION LOCAL  DEFAULT   33 .shstrtab
    34: 0000000000000000     0 SECTION LOCAL  DEFAULT   34 .debug_abbrev
    35: 0000000000000000     0 SECTION LOCAL  DEFAULT   35 .debug_aranges
    36: 0000000000000000     0 SECTION LOCAL  DEFAULT   36 .debug_info
    37: 0000000000000000     0 SECTION LOCAL  DEFAULT   37 .debug_line
    38: 0000000000000000     0 SECTION LOCAL  DEFAULT   38 .debug_line_str
    39: 0000000000000000     0 SECTION LOCAL  DEFAULT   39 .debug_rnglists
    40: 0000000000000000     0 SECTION LOCAL  DEFAULT   40 .debug_str
    41: 0000000000000000     0 SECTION LOCAL  DEFAULT   41 .strtab
    42: 0000000000000000     0 SECTION LOCAL  DEFAULT   42 .symtab
    43: 0000000000000000     0 SECTION LOCAL  DEFAULT   43 .gdb_index
    44: 0501001abc184bb7 0x28011b1d005871c0 <processor specific>: 13 WEAK   HIDDEN  [<other>: 58]  bad section index[7432] <corrupt>
    45: 1ace54001ace4401 0x540100201f400200 <processor specific>: 13 GLOBAL HIDDEN  [<other>: cc]    26 _referenceISS_E4typeE
    46: bb1d0064cbb22c00 0x2c01001abc444b OBJECT  <unknown>: 6 HIDDEN  [<other>: cc]    26 í^Bîq^A
    47: 02e5600e0b01001a 0x49f28f0901201d <processor specific>: 14 <OS specific>: 11 PROTECTED [<other>: 48]  bad section index[48279] <corrupt>
    48: 1d0100201f400200 0x1ace5401001ace NOTYPE  LOCAL  DEFAULT [<other>: 94]  bad section index[6862] <corrupt>
    49: ce540134ec7e2a01 0xb4001aceae01001a RELC    <processor specific>: 14 PROTECTED bad section index[10269] <corrupt>

[akarle]

Could the fix to the new corruption be as simple as moving compute_section_size and compute_section_headers back to below the wiping of sections (the way the were a few commits ago)? I'm taking an educated guess here, but computing the size and then overwriting the chunk with empty placeholders feels like it could be the cause of offset/corruption:

diff --git a/src/passes.cc b/src/passes.cc
index c7fb43b4..700a2148 100644
--- a/src/passes.cc
+++ b/src/passes.cc
@@ -3398,18 +3398,6 @@ void write_separate_debug_file(Context<E> &ctx) {
   i64 num_chunks = ctx.chunks.size();
   append(ctx.chunks, ctx.debug_chunks);
 
-  tbb::parallel_for(num_chunks, (i64)ctx.chunks.size(), [&](i64 i) {
-    ctx.chunks[i]->compute_section_size(ctx);
-  });
-
-  sort_debug_info_sections(ctx);
-
-  // Handle --compress-debug-info
-  if (ctx.arg.compress_debug_sections != ELFCOMPRESS_NONE)
-    compress_debug_sections(ctx);
-
-  // Recompute section header contents since we have added debug sections
-  compute_section_headers(ctx);
 
   // A debug info file contains all sections as the original file, though
   // most of them can be empty as if they were bss sections. We convert
@@ -3428,6 +3416,19 @@ void write_separate_debug_file(Context<E> &ctx) {
     }
   }
 
+  tbb::parallel_for(num_chunks, (i64)ctx.chunks.size(), [&](i64 i) {
+    ctx.chunks[i]->compute_section_size(ctx);
+  });
+
+  sort_debug_info_sections(ctx);
+
+  // Handle --compress-debug-info
+  if (ctx.arg.compress_debug_sections != ELFCOMPRESS_NONE)
+    compress_debug_sections(ctx);
+
+  // Recompute section header contents since we have added debug sections
+  compute_section_headers(ctx);
+
   // Assign file offsets to sections
   i64 fileoff = 0;
   for (Chunk<E> *chunk : ctx.chunks) {

Anecdotally, with the above patch, both our production binary and our repro don't show any signs of corruption, but I'm far too ignorant about the inner workings of the code to claim that it's the right fix :)

[akarle]

Hi Rui,

We found another failure mode that's notable / more concerning than just the gdb slowness--it looks like with -Wl,--separate-debug-file (in 2.40.3 / prior to the first attempted fix), gdb is unable to determine the right memory location for variables (likely due to the PHDR section / offsets). If I link with my proposed fix directly above on top of main, it's no longer an issue.

Here's a minimal repro:

$  cat main.c
#include <stdio.h>
extern int x;

int main() {
    if (x == 123) {
        puts("gdb changed this variable!");
    } else {
        puts("gdb failed to change this variable!");
    }
    return 0;
}

$ cat libx.c
int x = 456;


# it doesn't matter what the library is linked with, so choose ld to rule out mold
$ cc -g -O0 -c libx.c -o libx.o
$ cc -g -O0 -shared -o libx.so libx.o

# things work fine without -Wl,--separate-debug-file
$ cc -g -O0 -fuse-ld=mold -o good main.c -L. -lx -Wl,--rpath='$ORIGIN'
$ objcopy --only-keep-debug good good.dbg
$ strip good
$ objcopy --add-gnu-debuglink=good.dbg good

$ cc -g -O0 -fuse-ld=mold -o bad main.c -Wl,--separate-debug-file -L. -lx -Wl,--rpath='$ORIGIN'

The goal is to run something like this in gdb:

(gdb) break main
(gdb) run
(gdb) set x = 123
(gdb) continue

and hit the changed this variable branch. This works with good but not with bad.

The symtab/dynsym looks fine for the bad executable (x virtual address lines up):

$ readelf -a bad | grep x$
     4: 0000000000003a30     4 OBJECT  GLOBAL DEFAULT   30 x


# this matches the disassembled main:

$ gdb bad
(gdb) disassemble main
Dump of assembler code for function main:
   0x0000000000001779 <+0>:     push   %rbp
   0x000000000000177a <+1>:     mov    %rsp,%rbp
   0x000000000000177d <+4>:     mov    0x22ad(%rip),%eax        # 0x3a30 <x>

But with -Wl,--separate-debug-file, gdb has a bad address for x:

$ gdb bad
(gdb) p &x
$1 = (int *) 0x4428
(gdb) set x = 5
Cannot access memory at address 0x4428

The good executable (also mold, just without --separate-debug-file) works fine in gdb.