@Test
	public void threeModules() {
		Tokenizer t = new Tokenizer("foo | bar | goo");
		checkTokens(t, token_id("foo", 0, 3), token(PIPE, 4, 5), token_id("bar", 6, 9), token(PIPE, 10, 11),
				token_id("goo", 12, 15));
	}
Check the tokenizer breaks strings up into correct token streams.

@author Andy Clement
spring-xd-dirt/src/test/java/org/springframework/xd/dirt/stream/dsl/TokenizerTests.java