This section explains some basic aspects of lexer operation including buffer usage and matching behavior.