Class TokenManager

constructor

• new TokenManager(
      labelManager: LabelManager,
      tokens: TokenizerSpans[],
      currentParagraph?: null | Paragraph,
  ): TokenManager

  Constructor for TokenManager.

  Parameters

  • labelManager: LabelManager

    The label manager associated with the TokenManager.
  • tokens: TokenizerSpans[]

    An array of TokenizerSpans objects to initialize the tokens.
  • currentParagraph: null | Paragraph = null

    The current paragraph

  Returns TokenManager

  Description

  The constructor initializes the TokenManager with a LabelManager and an array of tokens.

edited

labelManager

tokens

tokenBlocks

• get tokenBlocks(): TMTokenBlock[]

  Returns TMTokenBlock[]

addBlockFromStructure

• addBlockFromStructure(entity: Entity | TMTokenBlock): void

  Adds a block from an Entity or TMTokenBlock structure.

  Parameters

  • entity: Entity | TMTokenBlock

    The Entity or TMTokenBlock structure to add.

  Returns void

  Description

  This method adds a new block to the TokenManager from an Entity or TMTokenBlock structure.

addNewBlock

• addNewBlock(
      start: number,
      end: number,
      labelClass: undefined | Label,
      currentState: string,
      history?: History[],
  ): void

  Adds a new block of tokens to the TokenManager.

  Parameters

  • start: number

    The start index of the new block.
  • end: number

    The end index of the new block.
  • labelClass: undefined | Label

    The label class for the new block, can be undefined.
  • currentState: string

    The current state of the new block.
  • Optionalhistory: History[] = []

    An optional array of history entries for the new block.

  Returns void

  Description

  This method adds a new block of tokens, handling overlapping blocks and updating the tokens array accordingly.

blocksInRange

• blocksInRange(start: number, end: number): TMTokens[]

  Gets all blocks in the specified range.

  Parameters

  • start: number

    The start index of the range.
  • end: number

    The end index of the range.

  Returns TMTokens[]

  • An array of TMTokens that overlap with the specified range.

  Description

  This method retrieves all blocks that overlap with the specified start and end indices.

getBlockByStart

• getBlockByStart(start: number): null | TMTokenBlock

  Gets a block by its start index.

  Parameters

  • start: number

    The start index of the block to retrieve.

  Returns null | TMTokenBlock

  • The block if found, otherwise null

  Description

  This method retrieves a block from the tokens array by its start index.

isOverlapping

• isOverlapping(start: number, end: number): null | TMTokens[]

  Checks if there are any overlapping blocks in the specified range.

  Parameters

  • start: number

    The start index of the range.
  • end: number

    The end index of the range.

  Returns null | TMTokens[]

  • An array of overlapping blocks if found, otherwise null.

  Description

  This method checks if there are any blocks that overlap with the specified start and end indices.

removeBlock

• removeBlock(start: number, reintroduceTokens?: boolean): void

  Removes a block of tokens by its start index.

  Parameters

  • start: number

    The start index of the block to remove.
  • OptionalreintroduceTokens: boolean = true

    Whether to reintroduce the tokens from the removed block back into the tokens array.

  Returns void

  Description

  This method removes a block of tokens from the TokenManager by its start index.

removeDuplicateBlocks

• removeDuplicateBlocks(): void

  Removes duplicate blocks from the tokens array.

  Returns void

  Description

  This method removes duplicate blocks from the tokens array, ensuring that each block is unique.

restoreOriginalBlockState

• restoreOriginalBlockState(start: number): void

  Restores the original state of a block by its start index.

  Parameters

  • start: number

    The start index of the block to restore.

  Returns void

  Description

  This method restores the original state of a block by its start index, resetting its properties to their original values.