Class SourceToken

Represents a smallest lexical unit of a source text

Inheritance
System.Object
SyntaxNode
SourceToken
Inherited Members
System.Object.Equals(System.Object)
System.Object.Equals(System.Object, System.Object)
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.GetHashCode()
System.Object.GetType()
System.Object.MemberwiseClone()
Namespace: CilTools.SourceCode.Common
Assembly: CilTools.SourceCode.dll
Syntax
public class SourceToken : SyntaxNode

Constructors

SourceToken(String, TokenKind)

Creates a new source token instance

Declaration
public SourceToken(string content, TokenKind kind)
Parameters
Type Name Description
System.String content
TokenKind kind

SourceToken(String, TokenKind, String, String)

Creates a new source token instance with whitespace

Declaration
public SourceToken(string content, TokenKind kind, string leadingWhitespace, string trailingWhitespace)
Parameters
Type Name Description
System.String content
TokenKind kind
System.String leadingWhitespace
System.String trailingWhitespace

Properties

Content

Gets the text content of this token without leading and trailing whitespace

Declaration
public string Content { get; }
Property Value
Type Description
System.String

Kind

Gets the token kind

Declaration
public TokenKind Kind { get; }
Property Value
Type Description
TokenKind

Language

Gets or sets the programming language name for the document this token is included in

Declaration
public string Language { get; set; }
Property Value
Type Description
System.String

OrdinalNumber

Gets or sets the ordinal number of this token in a sequence of tokens. The value is undefined if this token is not included in a sequence of tokens.

Declaration
public int OrdinalNumber { get; set; }
Property Value
Type Description
System.Int32
Remarks

This property could be useful to find previous/next token in a sequence of tokens parsed from a source text

Methods

EnumerateChildNodes()

Enumerates child nodes of this node. For the leaf node, returns an empty collection.

Declaration
public override IEnumerable<SyntaxNode> EnumerateChildNodes()
Returns
Type Description
System.Collections.Generic.IEnumerable<SyntaxNode>

The collection of child syntax nodes
Overrides

ToText(TextWriter)

Writes text representation of this node into the specified TextWriter

Declaration
public override void ToText(TextWriter target)
Parameters
Type Name Description
System.IO.TextWriter target
Overrides