Yudai Takada

Talk: How does Lrama make the Ruby parser grammar G.O.A.T.?

OSS programmer. Committer of Lrama. Member of RuboCop RSpec team. Maintainer of Committee. Co-Founder of Kyobashi.rb.

How does Lrama make the Ruby parser grammar G.O.A.T.?

The Ruby parser has historically been hacked in many complex ways. So, parse.y called Demon Castle. In Ruby 3.3, Lrama now gener- ates Ruby parsers instead of GNU Bison, which generated Ruby parser. I introduce how Lrama extends and improves the grammar of Ruby parser.

In this talk, I introduce how to make difficult Ruby parser grammar files (parse.y) easier to understand and what the future holds.

Until now, Ruby parser grammar files have been constrained by GNU Bison to represent flexible Ruby grammar through complex hacks. So, they have been very difficult to maintain, and there has been a lively discussion about the need to improve the maintainability, flexibility, and usability of the Ruby parser.

However, this changed with the replacement of GNU Bison with Lrama in Ruby 3.3. This makes it easier for us to extend richer grammars.

For example, the feature to concisely represent frequent patterns improves the readability of grammar files. Also, when adding or updating grammars in a grammar file, conflicts with existing grammars can be difficult to resolve. If you have a feature that allows you to easily resolve conflicts, you should be able to easily change the grammar.

The following are some of the major extensions I will cover in the talk:

Parameterizing Rules:

  • Provides notation for parameterizing the definition of a non-terminal sym- bol with other (terminal or non-terminal) symbols
  • Supports EBNF syntax, you can write grammar files using the ?, +, and * modifiers

Standard Library:

  • Provides standard libraries with standard, generic definitions of options, lists, etc.
  • Users can use these definitions to succinctly and effectively represent com- mon patterns

Inline:

  • Provides a solution to avoid conflicts without changing grammar
  • Conflicts are difficult to resolve, but they can be effectively handled with- out complex changes to parser.

I will explain the improvements that these extensions bring to Ruby parser’s grammar files with specific examples, and also cover the basics of parser and parser generator, and the current state of Ruby parser. Therefore, I plan to make the structure easy to understand for audiences who don’t know much about Ruby parser.

Bio

OSS programmer. Committer of Lrama. Member of RuboCop RSpec team. Maintainer of Committee. Co-Founder of Kyobashi.rb.