private static void writeTopN(Path path, long totalDocs, long sumDocFreqs, long sumTotalTermFreqs, long uniqueTerms, AbstractTokenTFDFPriorityQueue queue) throws IOException {
        if (Files.isRegularFile(path)) {
            System.err.println("File " + path.getFileName() + " already exists. Skipping.");
            return;
        }
        Files.createDirectories(path.getParent());
        try (BufferedWriter writer = Files.newBufferedWriter(path, StandardCharsets.UTF_8)) {
            StringBuilder sb = new StringBuilder();
            writer.write(LICENSE);
            writer.write("#DOC_COUNT\t" + totalDocs + "\n");
            writer.write("#SUM_DOC_FREQS\t" + sumDocFreqs + "\n");
            writer.write("#SUM_TERM_FREQS\t" + sumTotalTermFreqs + "\n");
            writer.write("#UNIQUE_TERMS\t" + uniqueTerms + "\n");
            writer.write("#TOKEN\tDOCFREQ\tTERMFREQ\n");
            //add these tokens no matter what
            for (String t : INCLUDE_LIST) {
                writer.write(t);
                writer.newLine();
            }
            for (TokenDFTF tp : queue.getArray()) {
                writer.write(getRow(sb, tp) + "\n");

            }
            writer.flush();
        }
    }
Utility class that reads in a UTF-8 input file with one document per row
and outputs the 20000 tokens with the highest document frequencies.
<p>
The CommmonTokensAnalyzer intentionally drops tokens shorter than 4 characters,
but includes bigrams for cjk.
<p>
It also has a include list for __email__ and __url__ and a skip list
for common html markup terms.
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/tools/TopCommonTokenCounter.java