@Test
	public void jobTokenization() {
		Tokens tokens = new Tokens("jobA | BROKEN = $END |'*'=something");
		assertToken(TokenKind.IDENTIFIER, "jobA", 0, 4, tokens.next());
		assertToken(TokenKind.PIPE, "|", 5, 6, tokens.next());
		assertToken(TokenKind.IDENTIFIER, "BROKEN", 7, 13, tokens.next());
		assertToken(TokenKind.EQUALS, "=", 14, 15, tokens.next());
		assertToken(TokenKind.IDENTIFIER, "$END", 16, 20, tokens.next());
		assertToken(TokenKind.PIPE, "|", 21, 22, tokens.next());
		assertToken(TokenKind.LITERAL_STRING, "'*'", 22, 25, tokens.next());
		assertToken(TokenKind.EQUALS, "=", 25, 26, tokens.next());
		assertToken(TokenKind.IDENTIFIER, "something", 26, 35, tokens.next());
	}
Parse job specifications and verify either the correct abstract syntax tree is produced or the current exception comes out.
The parser does no semantic validation, it is purely syntax checking.

@author Andy Clement
spring-xd-dirt/src/test/java/org/springframework/xd/dirt/job/dsl/JobParserTests.java