@Override
  public RecordReader<LongWritable, Text> getRecordReader(final InputSplit genericSplit, final JobConf job,
    final Reporter reporter)
    throws IOException {
    if (genericSplit instanceof FileSplit) {
      final Path finalPath = ((FileSplit) genericSplit).getPath();
      LOG.debug("Returning record reader for path {}", finalPath);
      jobConf = projectionPusher.pushProjectionsAndFilters(job, finalPath.getParent());
    }
    // textIF considers '\r' or '\n' as line ending but syslog uses '\r' for escaping new lines. So to read multi-line
    // exceptions correctly we explicitly use only '\n'
    jobConf.set("textinputformat.record.delimiter", "\n");
    return super.getRecordReader(genericSplit, jobConf, reporter);
  }
Syslog input format handles files written to sys.logs hive table. This input format can prune files based on
predicates in ts (timestamp) column. Following are the assumptions/supported options for the file pruning to work.
When configuring log shippers (like fluentd), the following directory layout has to be followed

WAREHOUSE_DIR/sys.db/logs/dt=DATE/ns=NAMESPACE/app=APPLICATION/YYYY-mm-DD-HH-MM_i[.log][.gz]
WAREHOUSE_DIR - is hive warehouse location for external tables
DATE - date of the log files
NAMESPACE - namespace for the applications
APPLICATION - name of the application (hiveserver2, metastore etc.)
YYYY - year
mm - month
DD - day of the month
HH - hour of the day (24 hr format)
MM - minute
i - index (log rotation)
[.log] - optional log extension
[.gz] - optional gz compression extension

When executing queries like
select count(*) from sys.logs where ts between current_timestamp() - INTERVAL '1' DAY and current_timestamp()

syslog input format can prunes the file listing based on the query filter condition.
File Pruning Assumptions/Support:
- All timestamps are in UTC (timestamp in file name, log lines and in query predicates all assume UTC)
- File names are in YYYY-mm-DD-HH-MM_i[.log][.gz] format.
- Seconds and millis in predicates gets rounded off to nearest minute (essentially ignoring them).
- Time slice used by log aggregation library should match with hive.syslog.input.format.file.time.slice config.
  If a filename is 2019-04-02-21-00_0.log.gz and timeslice is 300s then the file 2019-04-02-21-00_0.log.gz is
  expected to have log lines from timestamp 2019:04:02 21:00:00 to 2019:04:02 21:05:00 timestamp.
- Logs table should have 'ts' as timestamp column.
- Only simple BETWEEN filter predicate is supported for 'ts' column. There cannot be &gt;1 predicates on 'ts' column.
ql/src/java/org/apache/hadoop/hive/ql/log/syslog/SyslogInputFormat.java