Fix unsafe defaulted_state reductions and align with Bison defaulted_state reduction behavior #314

justincui · 2025-08-07T02:36:12Z

1 Update yacc.py
1.1 The original implementation did not prevent the defaulted_state from reducing in certain cases, which could lead to a deadlock when handling erroneous input. Although the condition if len(rules) == 1 and rules[0] < 0: was overly strict and made this scenario rare, it still posed a correctness risk.

1.2 Updated the defaulted_state rule to align with Bison’s behavior for handling default reductions, and removed unnecessary lookahead checks to simplify parser-initiated lexer state transitions.

2 Update tests/yacc_error5.py
2.1 Since we now prevent reductions from defaulted_state during error recovery, error-handling rules that lack a right-hand symbol boundary are no longer meaningful and should be reconsidered or removed.

…cing 1. The original implementation did not prevent the defaulted_state from reducing in certain cases, which could lead to a deadlock when processing erroneous input. Although the previous condition if len(rules) == 1 and rules[0] < 0: was overly strict and made such a scenario highly unlikely, it was not completely safe. 2. Updated the defaulted_state rule to align with Bison’s behavior for handling defaulted_state reductions, and removed unnecessary lookahead checks to simplify state transitions—making it easier for the parser to trigger lexer state changes.

Since we now prevent reductions from defaulted_state during error recovery, error-handling rules that lack a right-hand symbol boundary are no longer meaningful and should be reconsidered or removed.

dabeaz · 2025-08-07T14:19:10Z

Interesting. Do you have a small example of a grammar that illustrates this bug? I'm just trying to better understand it before committing to making changes.

justincui · 2025-08-08T02:29:37Z

Unfortunately, I’m currently unable to restore the setup needed to reproduce the error. Fortunately, I do have a log that shows exactly how the deadlock occurred.

Some background on my parser: I’m developing a parser for a configuration file with very messy and loosely defined syntax. This required me to switch lexer states within the parser. I had previously implemented this successfully in a Java-based Bison parser.

However, when I ported the implementation to PLY, I encountered an issue with an unwanted lookahead token—one that Bison would not normally look ahead for. After examining yacc.py, I discovered that the defaulted_state handling was different from Bison 3.3. I updated defaulted_state to match Bison’s behavior, but that change introduced a deadlock.

The attached log provides detailed traces of how the deadlock occurred. After further investigation, I realized that defaulted_state should not be used during error handling. I then fixed the issue accordingly, as shown in the pull request.

( Bison’s “default reduction” conditions:

It rules out shift/reduce conflicts: If a state contains both shifts (positive values) and reductions (negative values), the all() check will definitely fail because a positive value is not equal to a negative value.
It rules out reduce/reduce conflicts: If a state reduces to different rules depending on the lookahead token (e.g., action values -30 and -45), the all() check will likewise fail because -45 is not equal to -30.
It correctly identifies the target scenario: When a state’s only and definite action, regardless of which valid lookahead token appears, is to reduce by the same rule (for example, in state 8, all actions are -30), both conditions are satisfied and the state is correctly recognized.)

In this log, the earlier errors are all handled normally, but the last error leads to a deadlock due to the 'lex_state_skipLN' empty rule reducing based on defaulted_state. 'lex_state_skipLN' is used to change lexer state. (btw, L*T in the rules are keywords and operators like "var", "=")

State  : 303
Stack  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT L4T pogocabling_settings . LexToken(L3T,',',108,4212)
Action : Shift and goto state 364
State  : 364
Stack  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT L4T pogocabling_settings L3T . LexToken(INT,325,108,4214)
ERROR: Error  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT L4T pogocabling_settings L3T . LexToken(INT,325,108,4214)
Syntax error at [line 108, col 25]: 325 (unexpected 'INT', expect: {'Identifier'})
           allow_density_mismatch
           101: pogoblocks="125, 325, 126, 326"
         ........................^^^
State  : 364
Stack  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT L4T pogocabling_settings L3T . error
ERROR: Error  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT L4T pogocabling_settings L3T . error
State  : 303
Stack  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT L4T pogocabling_settings . error
ERROR: Error  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT L4T pogocabling_settings . error
State  : 247
Stack  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT L4T . error
ERROR: Error  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT L4T . error
State  : 191
Stack  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT . error
ERROR: Error  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling INT . error
State  : 134
Stack  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling . error
ERROR: Error  : model_module lex_section_pogocabling L44T pogocabling_body new_pogo_cabling . error
State  : 76
Stack  : model_module lex_section_pogocabling L44T pogocabling_body . error
ERROR: Error  : model_module lex_section_pogocabling L44T pogocabling_body . error
State  : 48
Stack  : model_module lex_section_pogocabling L44T . error
ERROR: Error  : model_module lex_section_pogocabling L44T . error
State  : 30
Stack  : model_module lex_section_pogocabling . error
ERROR: Error  : model_module lex_section_pogocabling . error
State  : 1
Stack  : model_module . error
ERROR: Error  : model_module . error
State  : 0
Stack  : . error
ERROR: Error  : . error
State  : 0
Stack  : . LexToken(L3T,',',108,4217)
ERROR: Error  : . LexToken(L3T,',',108,4217)
State  : 0
Stack  : . LexToken(INT,126,108,4219)
ERROR: Error  : . LexToken(INT,126,108,4219)
State  : 0
Stack  : . LexToken(L3T,',',108,4222)
ERROR: Error  : . LexToken(L3T,',',108,4222)
State  : 0
Stack  : . LexToken(INT,326,108,4224)
ERROR: Error  : . LexToken(INT,326,108,4224)
State  : 0
Stack  : . LexToken(STR,'\n  103: pogoblocks=',108,4227)
ERROR: Error  : . LexToken(STR,'\n  103: pogoblocks=',108,4227)
State  : 0
Stack  : . LexToken(INT,127,108,4248)
ERROR: Error  : . LexToken(INT,127,108,4248)
State  : 0
Stack  : . LexToken(L3T,',',108,4251)
ERROR: Error  : . LexToken(L3T,',',108,4251)
State  : 0
Stack  : . LexToken(INT,327,108,4253)
ERROR: Error  : . LexToken(INT,327,108,4253)
State  : 0
Stack  : . LexToken(L3T,',',108,4256)
ERROR: Error  : . LexToken(L3T,',',108,4256)
State  : 0
Stack  : . LexToken(INT,128,108,4258)
ERROR: Error  : . LexToken(INT,128,108,4258)
State  : 0
Stack  : . LexToken(L3T,',',108,4261)
ERROR: Error  : . LexToken(L3T,',',108,4261)
State  : 0
Stack  : . LexToken(INT,328,108,4263)
ERROR: Error  : . LexToken(INT,328,108,4263)
State  : 0
Stack  : . LexToken(STR,'\n  105: pogoblocks=',108,4266)
ERROR: Error  : . LexToken(STR,'\n  105: pogoblocks=',108,4266)
State  : 0
Stack  : . LexToken(INT,129,108,4287)
ERROR: Error  : . LexToken(INT,129,108,4287)
State  : 0
Stack  : . LexToken(L3T,',',108,4290)
ERROR: Error  : . LexToken(L3T,',',108,4290)
State  : 0
Stack  : . LexToken(INT,329,108,4292)
ERROR: Error  : . LexToken(INT,329,108,4292)
State  : 0
Stack  : . LexToken(L3T,',',108,4295)
ERROR: Error  : . LexToken(L3T,',',108,4295)
State  : 0
Stack  : . LexToken(INT,130,108,4297)
ERROR: Error  : . LexToken(INT,130,108,4297)
State  : 0
Stack  : . LexToken(L3T,',',108,4300)
ERROR: Error  : . LexToken(L3T,',',108,4300)
State  : 0
Stack  : . LexToken(INT,330,108,4302)
ERROR: Error  : . LexToken(INT,330,108,4302)
State  : 0
Stack  : . LexToken(UNKNOWN,'"\n\nBOARDCONF\n  1-8: type=DIG, core=FAST, mtype=PS1600, muxd = 1/2/3, clkd = 1, termd=1\n  PEX01: type=PEX, core=SLOW\n\nOSCILLOSCOPE\n  1: type=DSO, address=5025\n\nOPENNESSBOARD\n  MyOpenBoard1\n  MyOpenBoard2\n\namc\n  1: this=line, should=be, ignored=true\n\n',108,4305)
ERROR: Error  : . LexToken(UNKNOWN,'"\n\nBOARDCONF\n  1-8: type=DIG, core=FAST, mtype=PS1600, muxd = 1/2/3, clkd = 1, termd=1\n  PEX01: type=PEX, core=SLOW\n\nOSCILLOSCOPE\n  1: type=DSO, address=5025\n\nOPENNESSBOARD\n  MyOpenBoard1\n  MyOpenBoard2\n\namc\n  1: this=line, should=be, ignored=true\n\n',108,4305)
State  : 0
Stack  : . LexToken(NEWLINE,'\n\n',108,4306)
Action : Reduce rule [empty -> <empty>] with [] and goto state 3
Result : <str @ 0x7fa302725f70> ('')
State  : 3
Stack  : empty . LexToken(NEWLINE,'\n\n',108,4306)
Action : Reduce rule [model_head -> empty] with [''] and goto state 2
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 2
Stack  : model_head . LexToken(NEWLINE,'\n\n',108,4306)
Action : Shift and goto state 37
State  : 37
Stack  : model_head NEWLINE . LexToken(L6T,'BOARDCONF',110,4308)
Action : Reduce rule [model_head -> model_head NEWLINE] with [None,'\n\n'] and goto state 2
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 2
Stack  : model_head . LexToken(L6T,'BOARDCONF',110,4308)
Action : Reduce rule [model_module -> model_head] with [None] and goto state 1
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 1
Stack  : model_module . LexToken(L6T,'BOARDCONF',110,4308)
Action : Reduce rule [lex_section_boardconf -> <empty>] with [] and goto state 31
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 31
Stack  : model_module lex_section_boardconf . LexToken(L6T,'BOARDCONF',110,4308)
Action : Shift and goto state 49
State  : 49
Stack  : model_module lex_section_boardconf L6T . LexToken(NEWLINE,'\n',110,4317)
Action : Reduce rule [empty -> <empty>] with [] and goto state 79
Result : <str @ 0x7fa302725f70> ('')
State  : 79
Stack  : model_module lex_section_boardconf L6T empty . LexToken(NEWLINE,'\n',110,4317)
Action : Reduce rule [boardconf_body -> empty] with [''] and goto state 78
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 78
Stack  : model_module lex_section_boardconf L6T boardconf_body . LexToken(NEWLINE,'\n',110,4317)
Action : Shift and goto state 136
State  : 136
Stack  : model_module lex_section_boardconf L6T boardconf_body NEWLINE . LexToken(INT,1,111,4320)
Action : Reduce rule [boardconf_body -> boardconf_body NEWLINE] with [None,'\n'] and goto state 78
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 78
Stack  : model_module lex_section_boardconf L6T boardconf_body . LexToken(INT,1,111,4320)
Action : Reduce rule [new_board_conf -> <empty>] with [] and goto state 137
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 137
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf . LexToken(INT,1,111,4320)
Action : Shift and goto state 194
State  : 194
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf INT . LexToken(L5T,'-',111,4321)
Action : Shift and goto state 249
State  : 249
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf INT L5T . LexToken(INT,8,111,4322)
Action : Shift and goto state 313
State  : 313
Defaulted state 313: Reduce using 229
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf INT L5T INT . None
Action : Reduce rule [boardconf_line_index -> INT L5T INT] with [1,'-',8] and goto state 193
Result : <tuple @ 0x7fa2f03a19c0> ((1, 8))
State  : 193
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index . LexToken(L4T,':',111,4323)
Action : Shift and goto state 248
State  : 248
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T . LexToken(ID,'type',111,4325)
Action : Shift and goto state 310
State  : 310
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T ID . LexToken(L2T,'=',111,4329)
Action : Shift and goto state 367
State  : 367
Defaulted state 367: Reduce using 266
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T ID L2T . None
Action : Reduce rule [lex_state_rhs -> <empty>] with [] and goto state 397
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 397
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T ID L2T lex_state_rhs . LexToken(RHS_VALUE,'DIG',111,4330)
Action : Shift and goto state 420
State  : 420
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T ID L2T lex_state_rhs RHS_VALUE . LexToken(L3T,',',111,4333)
Action : Reduce rule [boardconf_setting -> ID L2T lex_state_rhs RHS_VALUE] with ['type','=',None,'DIG'] and goto state 309
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 309
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_setting . LexToken(L3T,',',111,4333)
Action : Reduce rule [boardconf_settings -> boardconf_setting] with [None] and goto state 307
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 307
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings . LexToken(L3T,',',111,4333)
Action : Shift and goto state 366
State  : 366
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T . LexToken(ID,'core',111,4335)
Action : Shift and goto state 310
State  : 310
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T ID . LexToken(L2T,'=',111,4339)
Action : Shift and goto state 367
State  : 367
Defaulted state 367: Reduce using 266
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T ID L2T . None
Action : Reduce rule [lex_state_rhs -> <empty>] with [] and goto state 397
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 397
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T ID L2T lex_state_rhs . LexToken(RHS_VALUE,'FAST',111,4340)
Action : Shift and goto state 420
State  : 420
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T ID L2T lex_state_rhs RHS_VALUE . LexToken(L3T,',',111,4344)
Action : Reduce rule [boardconf_setting -> ID L2T lex_state_rhs RHS_VALUE] with ['core','=',None,'FAST'] and goto state 396
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 396
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T boardconf_setting . LexToken(L3T,',',111,4344)
Action : Reduce rule [boardconf_settings -> boardconf_settings L3T boardconf_setting] with [None,',',None] and goto state 307
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 307
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings . LexToken(L3T,',',111,4344)
Action : Shift and goto state 366
State  : 366
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T . LexToken(ID,'mtype',111,4346)
Action : Shift and goto state 310
State  : 310
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T ID . LexToken(L2T,'=',111,4351)
Action : Shift and goto state 367
State  : 367
Defaulted state 367: Reduce using 266
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T ID L2T . None
Action : Reduce rule [lex_state_rhs -> <empty>] with [] and goto state 397
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 397
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T ID L2T lex_state_rhs . LexToken(RHS_VALUE,'PS1600',111,4352)
Action : Shift and goto state 420
State  : 420
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T ID L2T lex_state_rhs RHS_VALUE . LexToken(L3T,',',111,4358)
Action : Reduce rule [boardconf_setting -> ID L2T lex_state_rhs RHS_VALUE] with ['mtype','=',None,'PS1600'] and goto state 396
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 396
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T boardconf_setting . LexToken(L3T,',',111,4358)
Action : Reduce rule [boardconf_settings -> boardconf_settings L3T boardconf_setting] with [None,',',None] and goto state 307
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 307
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings . LexToken(L3T,',',111,4358)
Action : Shift and goto state 366
State  : 366
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T . LexToken(L7T,'muxd',111,4360)
Action : Shift and goto state 311
State  : 311
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T . LexToken(L2T,'=',111,4365)
Action : Shift and goto state 368
State  : 368
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T L2T . LexToken(INT,1,111,4367)
Action : Shift and goto state 399
State  : 399
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T L2T INT . LexToken(UNKNOWN,'/2/3, clkd = 1, termd=1\n  PEX01: type=PEX, core=SLOW\n\nOSCILLOSCOPE\n  1: type=DSO, address=5025\n\nOPENNESSBOARD\n  MyOpenBoard1\n  MyOpenBoard2\n\namc\n  1: this=line, should=be, ignored=true\n\n',111,4368)
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T L2T INT . LexToken(UNKNOWN,'/2/3, clkd = 1, termd=1\n  PEX01: type=PEX, core=SLOW\n\nOSCILLOSCOPE\n  1: type=DSO, address=5025\n\nOPENNESSBOARD\n  MyOpenBoard1\n  MyOpenBoard2\n\namc\n  1: this=line, should=be, ignored=true\n\n',111,4368)
Syntax error at [line 111, col 51]: '/2/3, clkd = 1, termd=1\n  PEX01: type=PEX, core=SLOW\n\nOSCILLOSCOPE\n  1: type=DSO, address=5025\n\nOPENNESSBOARD\n  MyOpenBoard1\n  MyOpenBoard2\n\namc\n  1: this=line, should=be, ignored=true\n\n' (unexpected 'UNKNOWN', expect: {',', '/', 'NEWLINE'})
         BOARDCONF
           1-8: type=DIG, core=FAST, mtype=PS1600, muxd = 1/2/3, clkd = 1, termd=1
         ..................................................^^^^^^^^^^^^^^^^^^^^^^^
State  : 399
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T L2T INT . error
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T L2T INT . error
State  : 368
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T L2T . error
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T L2T . error
State  : 311
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T . error
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T L7T . error
State  : 366
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T . error
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings L3T . error
State  : 307
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings . error
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T boardconf_settings . error
State  : 248
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T . error
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index L4T . error
State  : 193
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index . error
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body new_board_conf boardconf_line_index . error
State  : 137
Stack  : model_module lex_section_boardconf L6T boardconf_body new_board_conf . error
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body new_board_conf . error
State  : 78
Stack  : model_module lex_section_boardconf L6T boardconf_body . error
ERROR: Error  : model_module lex_section_boardconf L6T boardconf_body . error
State  : 49
Stack  : model_module lex_section_boardconf L6T . error
ERROR: Error  : model_module lex_section_boardconf L6T . error
State  : 31
Stack  : model_module lex_section_boardconf . error
ERROR: Error  : model_module lex_section_boardconf . error
State  : 1
Stack  : model_module . error
ERROR: Error  : model_module . error
State  : 0
Stack  : . error
ERROR: Error  : . error
State  : 0
Stack  : . LexToken(INT,2,111,4369)
ERROR: Error  : . LexToken(INT,2,111,4369)
State  : 0
Stack  : . LexToken(UNKNOWN,'/3, clkd = 1, termd=1\n  PEX01: type=PEX, core=SLOW\n\nOSCILLOSCOPE\n  1: type=DSO, address=5025\n\nOPENNESSBOARD\n  MyOpenBoard1\n  MyOpenBoard2\n\namc\n  1: this=line, should=be, ignored=true\n\n',111,4370)
ERROR: Error  : . LexToken(UNKNOWN,'/3, clkd = 1, termd=1\n  PEX01: type=PEX, core=SLOW\n\nOSCILLOSCOPE\n  1: type=DSO, address=5025\n\nOPENNESSBOARD\n  MyOpenBoard1\n  MyOpenBoard2\n\namc\n  1: this=line, should=be, ignored=true\n\n',111,4370)
State  : 0
Stack  : . LexToken(INT,3,111,4371)
ERROR: Error  : . LexToken(INT,3,111,4371)
State  : 0
Stack  : . LexToken(L3T,',',111,4372)
ERROR: Error  : . LexToken(L3T,',',111,4372)
State  : 0
Stack  : . LexToken(ID,'clkd',111,4374)
ERROR: Error  : . LexToken(ID,'clkd',111,4374)
State  : 0
Stack  : . LexToken(L2T,'=',111,4379)
ERROR: Error  : . LexToken(L2T,'=',111,4379)
State  : 0
Stack  : . LexToken(INT,1,111,4381)
ERROR: Error  : . LexToken(INT,1,111,4381)
State  : 0
Stack  : . LexToken(L3T,',',111,4382)
ERROR: Error  : . LexToken(L3T,',',111,4382)
State  : 0
Stack  : . LexToken(ID,'termd',111,4384)
ERROR: Error  : . LexToken(ID,'termd',111,4384)
State  : 0
Stack  : . LexToken(L2T,'=',111,4389)
ERROR: Error  : . LexToken(L2T,'=',111,4389)
State  : 0
Stack  : . LexToken(INT,1,111,4390)
ERROR: Error  : . LexToken(INT,1,111,4390)
State  : 0
Stack  : . LexToken(NEWLINE,'\n',111,4391)
Action : Reduce rule [empty -> <empty>] with [] and goto state 3
Result : <str @ 0x7fa302725f70> ('')
State  : 3
Stack  : empty . LexToken(NEWLINE,'\n',111,4391)
Action : Reduce rule [model_head -> empty] with [''] and goto state 2
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 2
Stack  : model_head . LexToken(NEWLINE,'\n',111,4391)
Action : Shift and goto state 37
State  : 37
Stack  : model_head NEWLINE . LexToken(ID,'PEX01',112,4394)
ERROR: Error  : model_head NEWLINE . LexToken(ID,'PEX01',112,4394)
State  : 37
Stack  : model_head NEWLINE . error
ERROR: Error  : model_head NEWLINE . error
State  : 2
Stack  : model_head . error
ERROR: Error  : model_head . error
State  : 0
Stack  : . error
ERROR: Error  : . error
State  : 0
Stack  : . LexToken(L4T,':',112,4399)
ERROR: Error  : . LexToken(L4T,':',112,4399)
State  : 0
Stack  : . LexToken(ID,'type',112,4401)
ERROR: Error  : . LexToken(ID,'type',112,4401)
State  : 0
Stack  : . LexToken(L2T,'=',112,4405)
ERROR: Error  : . LexToken(L2T,'=',112,4405)
State  : 0
Stack  : . LexToken(ID,'PEX',112,4406)
ERROR: Error  : . LexToken(ID,'PEX',112,4406)
State  : 0
Stack  : . LexToken(L3T,',',112,4409)
ERROR: Error  : . LexToken(L3T,',',112,4409)
State  : 0
Stack  : . LexToken(ID,'core',112,4411)
ERROR: Error  : . LexToken(ID,'core',112,4411)
State  : 0
Stack  : . LexToken(L2T,'=',112,4415)
ERROR: Error  : . LexToken(L2T,'=',112,4415)
State  : 0
Stack  : . LexToken(ID,'SLOW',112,4416)
ERROR: Error  : . LexToken(ID,'SLOW',112,4416)
State  : 0
Stack  : . LexToken(NEWLINE,'\n\n',112,4420)
Action : Reduce rule [empty -> <empty>] with [] and goto state 3
Result : <str @ 0x7fa302725f70> ('')
State  : 3
Stack  : empty . LexToken(NEWLINE,'\n\n',112,4420)
Action : Reduce rule [model_head -> empty] with [''] and goto state 2
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 2
Stack  : model_head . LexToken(NEWLINE,'\n\n',112,4420)
Action : Shift and goto state 37
State  : 37
Stack  : model_head NEWLINE . LexToken(L43T,'OSCILLOSCOPE',114,4422)
Action : Reduce rule [model_head -> model_head NEWLINE] with [None,'\n\n'] and goto state 2
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 2
Stack  : model_head . LexToken(L43T,'OSCILLOSCOPE',114,4422)
Action : Reduce rule [model_module -> model_head] with [None] and goto state 1
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 1
Stack  : model_module . LexToken(L43T,'OSCILLOSCOPE',114,4422)
Action : Reduce rule [lex_section_oscilloscope -> <empty>] with [] and goto state 32
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 32
Stack  : model_module lex_section_oscilloscope . LexToken(L43T,'OSCILLOSCOPE',114,4422)
Action : Shift and goto state 50
State  : 50
Stack  : model_module lex_section_oscilloscope L43T . LexToken(NEWLINE,'\n',114,4434)
Action : Reduce rule [empty -> <empty>] with [] and goto state 81
Result : <str @ 0x7fa302725f70> ('')
State  : 81
Stack  : model_module lex_section_oscilloscope L43T empty . LexToken(NEWLINE,'\n',114,4434)
Action : Reduce rule [oscilloscope_body -> empty] with [''] and goto state 80
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 80
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body . LexToken(NEWLINE,'\n',114,4434)
Action : Shift and goto state 139
State  : 139
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body NEWLINE . LexToken(INT,1,115,4437)
Action : Reduce rule [oscilloscope_body -> oscilloscope_body NEWLINE] with [None,'\n'] and goto state 80
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 80
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body . LexToken(INT,1,115,4437)
Action : Reduce rule [new_oscilloscope -> <empty>] with [] and goto state 140
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 140
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope . LexToken(INT,1,115,4437)
Action : Shift and goto state 197
State  : 197
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT . LexToken(L4T,':',115,4438)
Action : Shift and goto state 250
State  : 250
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T . LexToken(ID,'type',115,4440)
Action : Shift and goto state 317
State  : 317
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T ID . LexToken(L2T,'=',115,4444)
Action : Shift and goto state 371
State  : 371
Defaulted state 371: Reduce using 266
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T ID L2T . None
Action : Reduce rule [lex_state_rhs -> <empty>] with [] and goto state 402
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 402
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T ID L2T lex_state_rhs . LexToken(RHS_VALUE,'DSO',115,4445)
Action : Shift and goto state 422
State  : 422
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T ID L2T lex_state_rhs RHS_VALUE . LexToken(L3T,',',115,4448)
Action : Reduce rule [oscilloscope_setting -> ID L2T lex_state_rhs RHS_VALUE] with ['type','=',None,'DSO'] and goto state 316
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 316
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T oscilloscope_setting . LexToken(L3T,',',115,4448)
Action : Reduce rule [oscilloscope_settings -> oscilloscope_setting] with [None] and goto state 314
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 314
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T oscilloscope_settings . LexToken(L3T,',',115,4448)
Action : Shift and goto state 370
State  : 370
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T oscilloscope_settings L3T . LexToken(ID,'address',115,4450)
Action : Shift and goto state 317
State  : 317
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T oscilloscope_settings L3T ID . LexToken(L2T,'=',115,4457)
Action : Shift and goto state 371
State  : 371
Defaulted state 371: Reduce using 266
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T oscilloscope_settings L3T ID L2T . None
Action : Reduce rule [lex_state_rhs -> <empty>] with [] and goto state 402
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 402
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T oscilloscope_settings L3T ID L2T lex_state_rhs . LexToken(RHS_VALUE,'5025',115,4458)
Action : Shift and goto state 422
State  : 422
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T oscilloscope_settings L3T ID L2T lex_state_rhs RHS_VALUE . LexToken(NEWLINE,'\n\n',115,4462)
Action : Reduce rule [oscilloscope_setting -> ID L2T lex_state_rhs RHS_VALUE] with ['address','=',None,'5025'] and goto state 401
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 401
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T oscilloscope_settings L3T oscilloscope_setting . LexToken(NEWLINE,'\n\n',115,4462)
Action : Reduce rule [oscilloscope_settings -> oscilloscope_settings L3T oscilloscope_setting] with [None,',',None] and goto state 314
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 314
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body new_oscilloscope INT L4T oscilloscope_settings . LexToken(NEWLINE,'\n\n',115,4462)
Action : Reduce rule [oscilloscope_line -> new_oscilloscope INT L4T oscilloscope_settings] with [None,1,':',None] and goto state 138
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 138
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body oscilloscope_line . LexToken(NEWLINE,'\n\n',115,4462)
Action : Shift and goto state 196
State  : 196
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body oscilloscope_line NEWLINE . LexToken(L42T,'OPENNESSBOARD',117,4464)
Action : Reduce rule [oscilloscope_body -> oscilloscope_body oscilloscope_line NEWLINE] with [None,None,'\n\n'] and goto state 80
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 80
Stack  : model_module lex_section_oscilloscope L43T oscilloscope_body . LexToken(L42T,'OPENNESSBOARD',117,4464)
Action : Reduce rule [oscilloscope_section -> lex_section_oscilloscope L43T oscilloscope_body] with [None,'OSCILLOSCOPE',None] and goto state 17
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 17
Stack  : model_module oscilloscope_section . LexToken(L42T,'OPENNESSBOARD',117,4464)
Action : Reduce rule [section -> oscilloscope_section] with [None] and goto state 4
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 4
Stack  : model_module section . LexToken(L42T,'OPENNESSBOARD',117,4464)
Action : Reduce rule [model_module -> model_module section] with [None,None] and goto state 1
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 1
Stack  : model_module . LexToken(L42T,'OPENNESSBOARD',117,4464)
Action : Reduce rule [lex_section_opennessboard -> <empty>] with [] and goto state 33
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 33
Stack  : model_module lex_section_opennessboard . LexToken(L42T,'OPENNESSBOARD',117,4464)
Action : Shift and goto state 51
State  : 51
Stack  : model_module lex_section_opennessboard L42T . LexToken(NEWLINE,'\n',117,4477)
Action : Reduce rule [empty -> <empty>] with [] and goto state 83
Result : <str @ 0x7fa302725f70> ('')
State  : 83
Stack  : model_module lex_section_opennessboard L42T empty . LexToken(NEWLINE,'\n',117,4477)
Action : Reduce rule [opennessboard_body -> empty] with [''] and goto state 82
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 82
Stack  : model_module lex_section_opennessboard L42T opennessboard_body . LexToken(NEWLINE,'\n',117,4477)
Action : Shift and goto state 142
State  : 142
Stack  : model_module lex_section_opennessboard L42T opennessboard_body NEWLINE . LexToken(ID,'MyOpenBoard1',118,4480)
Action : Reduce rule [opennessboard_body -> opennessboard_body NEWLINE] with [None,'\n'] and goto state 82
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 82
Stack  : model_module lex_section_opennessboard L42T opennessboard_body . LexToken(ID,'MyOpenBoard1',118,4480)
Action : Reduce rule [new_openness_board -> <empty>] with [] and goto state 143
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 143
Stack  : model_module lex_section_opennessboard L42T opennessboard_body new_openness_board . LexToken(ID,'MyOpenBoard1',118,4480)
Action : Shift and goto state 199
State  : 199
Defaulted state 199: Reduce using 255
Stack  : model_module lex_section_opennessboard L42T opennessboard_body new_openness_board ID . None
Action : Reduce rule [opennessboard_line -> new_openness_board ID] with [None,'MyOpenBoard1'] and goto state 141
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 141
Stack  : model_module lex_section_opennessboard L42T opennessboard_body opennessboard_line . LexToken(NEWLINE,'\n',118,4492)
Action : Shift and goto state 198
State  : 198
Stack  : model_module lex_section_opennessboard L42T opennessboard_body opennessboard_line NEWLINE . LexToken(ID,'MyOpenBoard2',119,4495)
Action : Reduce rule [opennessboard_body -> opennessboard_body opennessboard_line NEWLINE] with [None,None,'\n'] and goto state 82
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 82
Stack  : model_module lex_section_opennessboard L42T opennessboard_body . LexToken(ID,'MyOpenBoard2',119,4495)
Action : Reduce rule [new_openness_board -> <empty>] with [] and goto state 143
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 143
Stack  : model_module lex_section_opennessboard L42T opennessboard_body new_openness_board . LexToken(ID,'MyOpenBoard2',119,4495)
Action : Shift and goto state 199
State  : 199
Defaulted state 199: Reduce using 255
Stack  : model_module lex_section_opennessboard L42T opennessboard_body new_openness_board ID . None
Action : Reduce rule [opennessboard_line -> new_openness_board ID] with [None,'MyOpenBoard2'] and goto state 141
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 141
Stack  : model_module lex_section_opennessboard L42T opennessboard_body opennessboard_line . LexToken(NEWLINE,'\n\n',119,4507)
Action : Shift and goto state 198
State  : 198
Stack  : model_module lex_section_opennessboard L42T opennessboard_body opennessboard_line NEWLINE . LexToken(L39T,'amc',121,4509)
Action : Reduce rule [opennessboard_body -> opennessboard_body opennessboard_line NEWLINE] with [None,None,'\n\n'] and goto state 82
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 82
Stack  : model_module lex_section_opennessboard L42T opennessboard_body . LexToken(L39T,'amc',121,4509)
Action : Reduce rule [opennessboard_section -> lex_section_opennessboard L42T opennessboard_body] with [None,'OPENNESSBOARD',None] and goto state 18
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 18
Stack  : model_module opennessboard_section . LexToken(L39T,'amc',121,4509)
Action : Reduce rule [section -> opennessboard_section] with [None] and goto state 4
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 4
Stack  : model_module section . LexToken(L39T,'amc',121,4509)
Action : Reduce rule [model_module -> model_module section] with [None,None] and goto state 1
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 1
Stack  : model_module . LexToken(L39T,'amc',121,4509)
Action : Shift and goto state 34
State  : 34
Stack  : model_module L39T . LexToken(NEWLINE,'\n',121,4512)
Action : Reduce rule [empty -> <empty>] with [] and goto state 53
Result : <str @ 0x7fa302725f70> ('')
State  : 53
Stack  : model_module L39T empty . LexToken(NEWLINE,'\n',121,4512)
Action : Reduce rule [obsolete_body -> empty] with [''] and goto state 52
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 52
Stack  : model_module L39T obsolete_body . LexToken(NEWLINE,'\n',121,4512)
Action : Shift and goto state 85
State  : 85
Stack  : model_module L39T obsolete_body NEWLINE . LexToken(INT,1,122,4515)
Action : Reduce rule [obsolete_body -> obsolete_body NEWLINE] with [None,'\n'] and goto state 52
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 52
Stack  : model_module L39T obsolete_body . LexToken(INT,1,122,4515)
Action : Shift and goto state 88
State  : 88
Defaulted state 88: Reduce using 265
Stack  : model_module L39T obsolete_body INT . None
Action : Reduce rule [obsolete_line_index -> INT] with [1] and goto state 86
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 86
Stack  : model_module L39T obsolete_body obsolete_line_index . LexToken(L4T,':',122,4516)
Action : Shift and goto state 145
State  : 145
Defaulted state 145: Reduce using 268
Stack  : model_module L39T obsolete_body obsolete_line_index L4T . None
Action : Reduce rule [lex_state_skipLN -> <empty>] with [] and goto state 200
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 200
Stack  : model_module L39T obsolete_body obsolete_line_index L4T lex_state_skipLN . LexToken(NEWLINE,'\n\n',122,4552)
ERROR: Error  : model_module L39T obsolete_body obsolete_line_index L4T lex_state_skipLN . LexToken(NEWLINE,'\n\n',122,4552)
Syntax error at [line 122, col 40]: '\n\n' (unexpected 'NEWLINE', expect: {'RHS_VALUE'})
         amc
           1: this=line, should=be, ignored=true
         .......................................|
State  : 200
Stack  : model_module L39T obsolete_body obsolete_line_index L4T lex_state_skipLN . error
ERROR: Error  : model_module L39T obsolete_body obsolete_line_index L4T lex_state_skipLN . error
State  : 145
Defaulted state 145: Reduce using 268
Stack  : model_module L39T obsolete_body obsolete_line_index L4T . error
Action : Reduce rule [lex_state_skipLN -> <empty>] with [] and goto state 200
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 200
Stack  : model_module L39T obsolete_body obsolete_line_index L4T lex_state_skipLN . error
ERROR: Error  : model_module L39T obsolete_body obsolete_line_index L4T lex_state_skipLN . error
State  : 145
Defaulted state 145: Reduce using 268
Stack  : model_module L39T obsolete_body obsolete_line_index L4T . error
Action : Reduce rule [lex_state_skipLN -> <empty>] with [] and goto state 200
Result : <NoneType @ 0x7fa3027094f0> (None)
State  : 200
Stack  : model_module L39T obsolete_body obsolete_line_index L4T lex_state_skipLN . error

justincui added 2 commits August 6, 2025 19:26

Update yacc_error5.py to remove the invalid error handling rule.

51956ad

Since we now prevent reductions from defaulted_state during error recovery, error-handling rules that lack a right-hand symbol boundary are no longer meaningful and should be reconsidered or removed.

justincui changed the title ~~Prevent unsafe defaulted_state reductions and align with Bison error recovery behavior~~ Fix unsafe defaulted_state reductions and align with Bison defaulted_state reduction behavior Aug 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix unsafe defaulted_state reductions and align with Bison defaulted_state reduction behavior #314

Fix unsafe defaulted_state reductions and align with Bison defaulted_state reduction behavior #314

Uh oh!

justincui commented Aug 7, 2025

Uh oh!

dabeaz commented Aug 7, 2025

Uh oh!

justincui commented Aug 8, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Fix unsafe defaulted_state reductions and align with Bison defaulted_state reduction behavior #314

Are you sure you want to change the base?

Fix unsafe defaulted_state reductions and align with Bison defaulted_state reduction behavior #314

Uh oh!

Conversation

justincui commented Aug 7, 2025

Uh oh!

dabeaz commented Aug 7, 2025

Uh oh!

justincui commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

justincui commented Aug 8, 2025 •

edited

Loading