Class TokenReader

Reads raw tokens from a string using the specified collection of token definitions

Inheritance

System.Object

TokenReader

Inherited Members

System.Object.ToString()

System.Object.Equals(System.Object)

System.Object.Equals(System.Object, System.Object)

System.Object.ReferenceEquals(System.Object, System.Object)

System.Object.GetHashCode()

System.Object.GetType()

System.Object.MemberwiseClone()

Namespace: CilTools.Syntax.Tokens

Assembly: CilTools.BytecodeAnalysis.dll

Syntax

public class TokenReader

Constructors

TokenReader(String, IEnumerable<SyntaxTokenDefinition>)

Creates a new instance of the token reader

Declaration

public TokenReader(string src, IEnumerable<SyntaxTokenDefinition> tokenDefinitions)

Parameters

Type	Name	Description
System.String	src	Source string
System.Collections.Generic.IEnumerable<SyntaxTokenDefinition>	tokenDefinitions	Collection of token definitions to use

Properties

Length

Gets the number of characters in the source string

Declaration

public int Length { get; }

Property Value

Type	Description
System.Int32

Position

Gets the position in the source string at which the next token would be read

Declaration

public int Position { get; }

Property Value

Type	Description
System.Int32

Methods

GetPreviousChar(Int32)

Gets a character at the specified offset before the current position in this token reader. Offset 0 means current position, offset 1 means character immediately before current position, etc. Does not change the current position.

Declaration

public char GetPreviousChar(int offset)

Parameters

Type	Name	Description
System.Int32	offset	The offset of the character to return

Returns

Type	Description
System.Char	Previous character at the specified offset, or zero if it is outside of the source string bounds.

GetSourceString()

Gets the full source string used by this reader

Declaration

public string GetSourceString()

Returns

Type	Description
System.String

PeekChar()

Gets a next character in this token reader without advancing a current position

Declaration

public char PeekChar()

Returns

Type	Description
System.Char	Next character in the current position, or zero if the end of string is reached.

PeekString(Int32)

Gets a specified number of next characters in this token reader without advancing a current position

Declaration

public string PeekString(int n)

Parameters

Type	Name	Description
System.Int32	n	Number of characters to peek

Returns

Type	Description
System.String	String containing n next characters, or an empty string when the end of string is reached.

ReadAll()

Reads all tokens from the current instance

Declaration

public IEnumerable<string> ReadAll()

Returns

Type	Description
System.Collections.Generic.IEnumerable<System.String>	Collection of strings that contain tokens