Parsing ​

Tree-sitter, Rust Sitter & Logos are neat. participle is great Go lib I use for parsing. nom, combine & LALRPOP are great for parsing in Rust.

Intro to Parsing with Parsec in Haskell is nice read.

basic-toml has nice parsing code in Rust. Lezer is interesting TS parser library.

Links ​

• Crowdsourcing the evolution of text parsing with unified (2018)
• Angstrom - Parser combinators built for speed and memory efficiency.
• tree-sitter - Parser generator tool and an incremental parsing library. (Web) (HN)
• RBNF.jl - DSL for modern parsing.
• Parse, don’t validate (2019) (Lobsters) (HN) (HN) (HN)
• DDL - Declarative data definition language for formally specifying binary data formats.
• JSON Parser with JavaScript (HN)
• Parsing 18 billion JSON lines with Go (2019)
• Parsing JSON Really Quickly: Lessons Learned (2019) (Article)
• Functional Parsing - Computerphile (2020)
• Common Expression Language - Implements common semantics for expression evaluation, enabling different applications to more easily interoperate. (HN)
• Awesome Unified
• Essentials of Interpretation (2020) (HN)
• pest - General purpose parser written in Rust with a focus on accessibility, correctness, and performance. (Web) (Lobsters)
• Simple but Powerful Pratt Parsing (2020)
• Reasoning About Recursive Tree Traversals (2019)
• unist - Specification for syntax trees.
• Essentials of Parsing (2020)
• Port of Rob Pike's Go lexer to Rust
• nearley - Streaming parser with support for catching errors gracefully and providing all parsings for ambiguous grammars.
• SwiftSoup - Pure Swift HTML Parser, with best of DOM, CSS, and jquery (Supports Linux, iOS, Mac, tvOS, watchOS).
• GoGLL - Generates a GLL parser and FSA-based lexer for any context-free grammar. The generated code is Go or Rust.
• How to Write a Lexer in Go (2020)
• Spicy - C++ parser generator that makes it easy to create robust parsers for network protocols, file formats, and more.
• Don't Panic! Better, Fewer, Syntax Errors for LR Parsers (2020) (Lobsters) (HN)
• Parsing: a timeline (2014) (HN)
• Which Parsing Approach? (2020) (Lobsters) (Reddit)
• Lark - Parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.
• Challenging LR Parsing (2020) (HN) (Lobsters)
• Why I write recursive descent parsers (despite their issues) (2020)
• mpc - Lightweight and powerful Parser Combinator library for C.
• Pure AST based linting sucks (2020)
• Awesome binary parsing - List of generic tools for parsing binary data structures, such as file formats, network protocols or bitstreams.
• Parsing with Scannerless Earley Virtual Machines (2019)
• North - Experimental SEVM (Scannerless Earley Virtual Machine) parser implementation.
• Just write the parser - Whirlwind tour of writing parsers by hand. (HN)
• Parsing Algorithms course (HN) (Code)
• Intro to Parsing with Parsec in Haskell - Demonstrates the basics of Parsec and goes on to build a SQL query parser.
• Parser Generator DSLs (2020)
• AST Explorer - Web tool to explore the ASTs generated by various parsers. (Code) (Web App Code)
• Lezer - Incremental GLR parser intended for use in an editor or similar system, which needs to keep a representation of the program current during changes and in the face of syntax errors. (GitHub) (Code)
• GraphRedex - Open-source tool that empowers language designers to interactively explore their reduction graphs.
• Parsimmon - Small library for writing big parsers made up of lots of little parsers.
• Inline caching: quickening (2021) (HN)
• Earley Parser (2021) (Lobsters)
• awesome syntax-tree - Curated list of awesome syntax-tree, unist, hast, xast, mdast, nlcst resources.
• Top-Down Parsing with Parsing Contexts -- A simple approach to context sensitive parsing (2014)
• Applicative parsing (2019)
• Parser Combinators From Scratch (2020) (Code)
• Ohm - Parser generator consisting of a library and a domain-specific language. (Code) (HN)
• Pika parsing: reformulating packrat parsing as a dynamic programming algorithm solves the left recursion and error recovery problems (2020)
• Sparser - Framework of various language parsers. (Web)
• Parsers all the way down: writing a self-hosting parser (2021)
• Canopy - Parser compiler for Java, JavaScript, Python, Ruby. (Docs)
• Principled Procedural Parsing (2019)
• Parsix - Parse, don't validate. (HN)
• parsegen - LR parser generator, similar to happy, ocamlyacc, and lalrpop.
• TextMapper - Lexer and Parser generator. (Web)
• Waxeye Parser Generator - Parser generator based on parsing expression grammars (PEGs).
• rust-peg - Parsing Expression Grammar (PEG) parser generator for Rust.
• Rust - Writing Parsers With nom Parser Combinator Framework (2021)
• GrammKit - Tool for generating syntax diagrams (also known as railroad diagrams) for parser grammars. (Code)
• PyParsing - Python library for creating PEG parsers.
• On Parsers (Lobsters)
• Casual Parsing in JavaScript (2021)
• Parser generators vs. handwritten parsers: surveying major language implementations in 2021 (HN) (Lobsters)
• Kaitai Struct - Declarative binary format parsing language. (Lobsters) (HN) (IDE) (HN)
• Parser toolkit - Toolkit that makes it easier to write recursive-descent parsers in Zig.
• TatSu - Generates Python parsers from grammars in a variation of EBNF. (HN)
• Progger - Simple set of parser and lexer tools. Built on top of logos and bumpalo.
• Types of Parser Combinators (2021) (Lobsters) (HN)
• Parsing Permutations (2021)
• Tree Sitter and the Complications of Parsing Languages (HN)
• Linguistics Tree Solver - Tool automatically builds linguistics syntax trees. (Code)
• Practical parsing with Flex and Bison (2021)
• A map of the tree-sitter ecosystem (2021)
• Getting started with tree-sitter (2021)
• Participle - Dead simple parser package for Go.
• Genre as Weak Supervision for Cross-lingual Dependency Parsing (2021) (Code)
• rd-parse - Generic minimalist zero dependencies recursive-descent parser generator in JavaScript.
• Lightweight Linting with Tree-sitter (2022)
• tsls - Tree-sitter based language server for general languages.
• Interactive tool for experimenting with LR(1) grammars
• syntax - Syntactic analysis toolkit, language-agnostic parser generator.
• ast-builder - Build your Abstract Syntax Trees (AST) directly by writing code.
• Packrat Parsing and Parsing Expression Grammars
• Invertible Parsing: The Problem (2022)
• Tree-sitter ｜ Code Navigation Systems (Tweet)
• Parsing Bitstreams with Rust and Nom (2022) (HN)
• Parsing Text with Nom (2022)
• How to write a tree-sitter grammar in an afternoon (2022)
• tree-sitter-comment - Tree-sitter grammar for comment tags like TODO, FIXME(user).
• Santiago - Lexing and parsing toolkit for Rust.
• Parsing Expressions by Recursive Descent (1999)
• From Precedence Climbing to Pratt Parsing (2016)
• Favorite approach to lexical analysis (i.e. turning a stream of bytes/characters into a stream of tokens)
• How to write a linter using tree-sitter in an hour (2022) (HN)
• ppx_match_seq - Syntax for Seq.t parsers.
• Tour of Parser-Printers: Introduction (2022) (HN)
• py-tree-sitter - Python bindings to the Tree-sitter parsing library.
• Handwritten Parsers & Lexers in Go (2014) (Lobsters)
• Binary build of tree-sitter for apple platforms
• TSPM - Open-source tree-sitter package manager.
• Lola - LL parser table generator.
• Parsing / Recursive Descent Parser (2022) (HN) (Lobsters)
• Parsing expressions by precedence climbing (2012) (HN)
• From recursive descent to LR parsing (2021)
• Pegen - Parser generator used in CPython to produce the parser used by the interpreter. It allows to produce PEG parsers from a description of a formal Grammar.
• lezer/generator - Parser generator for the lezer incremental parser.
• tree-sitter-viewer - Rust lib to visualize tree-sitter syntax tree.
• go tree-sitter - Go bindings for tree-sitter.
• tree-sitter-embedded-template - Tree-sitter grammar for embedded template languages like ERB, EJS.
• Interactive Parser Explanations
• Rust Sitter - Use Tree Sitter to parse your own languages in Rust. (Reddit) (HN)
• Parser Combinators and Interpreting a PL (2022)
• LALRPOP - LR(1) parser generator for Rust.
• Yap - Parser library for Rust.
• pratt - General Purpose Pratt Parser for Rust.
• Let's write a JSON parser in Rust
• Katydid - Encoding Agnostic Validation (Validation Language, Parsers and Encoders).
• Practical Parsing in Rust with nom (2022) (Lobsters)
• lexgen - Fully-featured lexer generator, implemented as a proc macro.
• parol - LL(k) parser generator for Rust.
• Parsing Programming Languages with FParsec (2020)
• What are the advantages of pratt parsing over recursive descent parsing? (2022)
• Case study of building parser combinators (Code)
• LL(1) Parser Visualization (HN)
• Quercus - Rich, ergonomic parsing library based on Tree-sitter.
• lang-pt - Language parser tool to build recursive descent top down parser.
• Generalized LL (GLL) Parser (2022)
• Tree Sitter Grammar.js LSP
• Faster general parsing through context-free memoization (2020)
• TreeStand - High-level Ruby wrapper for tree-sitter bindings.
• A Comprehensive Introduction to Tree-sitter (2021)
• Hammer Primer - Introduction to using the Hammer parser construction toolkit by Meredith L. Patterson.
• Why We Need to Know LR and Recursive Descent Parsing Techniques (2023) (HN) (Lobsters)
• An overview of parsing algorithms (2021)
• Tree Sitter Module - Building script for tree-sitter language definitions.
• Recursive-Descent Syntax for Context-Free Grammars (2023) (Reddit)
• Topiary - Uniform formatter for simple languages, as part of the Tree-sitter ecosystem. (Article)
• HOP - Lightweight, higher-order parser in Rust.
• Faster-Pest - High-performance code generator for Parsing Expression Grammars.
• Write better parsers with Nom Supreme (2023)
• Parsing TFTP in Rust (2022)
• PickTok - Parser combinator like nom but specialized in parsing &[Token].
• Write a JavaScript Parser in Rust (Code)
• peberminta - Simple, transparent parser combinators toolkit that supports tokens of any type.
• tspc - Strongly-typed parser combinator written in TypeScript.
• Parsinator - Build small well-defined parsers in TypeScript.
• ts-parsec - Parser combinator for TypeScript.
• Top-Down LR Parsing (2023) (HN)
• Valiant's Parser (2023)
• lezer/highlight - Syntax highlighting from Lezer trees.
• Optimizing a parser/compiler with data-oriented design: a case study (2023) (Lobsters)
• Parslet - Small PEG based parser library.
• code_tokenize - Fast tokenization and structural analysis of any programming language.
• flap: A Deterministic Parser with Fused Lexing (2023) (Lobsters)
• flap - Deterministic parser with fused lexing.
• tree-splicer - Simple grammar-based test case generator.
• Why Split Lexing and Parsing Into Two Separate Phases? (2023) (Lobsters)
• Parsing with Zippers
• Constructing human-grade parsers (2018)
• My Interview with Eelco Visser on Parsing (2023)
• astmaker - Build Abstract Syntax Trees and tree-walking models quickly in Rust. (Article) (HN)
• Resilient LL Parsing Tutorial (2023) (Code)
• automata-golf - Domain-specific language (DSL) for parsing regular, context-free and recursively enumerable languages.
• Grove - Universal AST parser built on Tree-sitter.
• rd - Build recursive descent parsers.
• How to Get Started with Tree-Sitter (HN)
• Peginator - PEG parser generator for creating ASTs in Rust.
• node-ebnf - Create AST PEG Parsers from formal grammars in JavaScript.
• Demystifying Pratt Parsers (2023) (Lobsters)
• Parlay - Editor engine based on parsing.
• Which Parsing Approach? (2020)
• Better code highlighting on the web (2023)
• rehype-tree-sitter - Rehype plugin to use tree-sitter to highlight code in <pre><code> blocks.
• LL(1) Parser Generator
• Ok I tried Tree-sitter! Here is what I think about it... (2023)
• tree-sitter-c2rust - Fork of Tree Sitter with a pure Rust runtime for WASM support.
• When URL parsers disagree (2023) (HN)