private static List<Token> tokenize(String string, char delimiter) {
    final List<Token> tokens = new ArrayList<>();
    final StringBuilder builder = new StringBuilder();
    for (int cursor = 0; cursor < string.length(); ) {
      final char c = string.charAt(cursor);

      int nextChar = cursor + 1;
      if (c == '\'') {
        nextChar = consumeInQuotes(string, '\'', cursor, builder);
        tokens.add(new Token(TokenType.SINGLE_QUOTED, builder.toString(), cursor));
      } else if (c == '"') {
        nextChar = consumeInQuotes(string, '"', cursor, builder);
        tokens.add(new Token(TokenType.DOUBLE_QUOTED, builder.toString(), cursor));
      } else if (c == delimiter) {
        tokens.add(new Token(TokenType.DELIMITER, String.valueOf(c), cursor));
      } else if (!Character.isWhitespace(c)) {
        nextChar = consumeUnquoted(string, delimiter, cursor, builder);
        tokens.add(new Token(TokenType.UNQUOTED, builder.toString().trim(), cursor));
      }
      builder.setLength(0);
      cursor = nextChar;
    }

    return tokens;
  }
Escapes the given string with single quotes, if the input string contains a double quote or any
of the given {@code charsToEscape}. Any single quotes in the input string will be escaped by
doubling.

<p>Given that the escapeChar is (;)

<p>Examples:

<ul>
  <li>A,B,C,D => A,B,C,D
  <li>A'B'C'D => 'A''B''C''D'
  <li>A;BCD => 'A;BCD'
  <li>AB"C"D => 'AB"C"D'
  <li>AB'"D:B => 'AB''"D:B'
</ul>

@param string a string which needs to be escaped
@param charsToEscape escape chars for the escape conditions
@return escaped string by single quote
amoro-common/src/main/java/org/apache/amoro/config/StructuredOptionsSplitter.java