public static void main(String[] args) throws Exception {
        Path commonTokensFile = Paths.get(args[0]);
        List<Path> inputFiles = new ArrayList<>();
        for (int i = 1; i < args.length; i++) {
            inputFiles.add(Paths.get(ProcessUtils.unescapeCommandLine(args[i])));
        }
        TopCommonTokenCounter counter = new TopCommonTokenCounter();
        if (Files.exists(commonTokensFile)) {
            System.err.println(commonTokensFile
                    .getFileName()
                    .toString() + " exists. I'm skipping this.");
            return;
        }
        counter.execute(commonTokensFile, inputFiles);
    }
Utility class that reads in a UTF-8 input file with one document per row
and outputs the 20000 tokens with the highest document frequencies.
<p>
The CommmonTokensAnalyzer intentionally drops tokens shorter than 4 characters,
but includes bigrams for cjk.
<p>
It also has a include list for __email__ and __url__ and a skip list
for common html markup terms.
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/tools/TopCommonTokenCounter.java