@Override
  public boolean next(NullWritable key, VectorizedRowBatch value) throws IOException {
    try {
      // Check and update partition cols if necessary. Ideally, this should be done
      // in CreateValue as the partition is constant per split. But since Hive uses
      // CombineHiveRecordReader and
      // as this does not call CreateValue for each new RecordReader it creates, this check is
      // required in next()
      if (addPartitionCols) {
        if (partitionValues != null) {
          rbCtx.addPartitionColsToBatch(value, partitionValues);
        }
        addPartitionCols = false;
      }
      if (!baseReader.next(null, vectorizedRowBatchBase)) {
        return false;
      }
    } catch (Exception e) {
      throw new IOException("error iterating", e);
    }
    if(!includeAcidColumns) {
      //if here, we don't need to filter anything wrt acid metadata columns
      //in fact, they are not even read from file/llap
      value.size = vectorizedRowBatchBase.size;
      value.selected = vectorizedRowBatchBase.selected;
      value.selectedInUse = vectorizedRowBatchBase.selectedInUse;
      copyFromBase(value);

      if (rowIsDeletedProjected) {
        rowIsDeletedVector.fill(false);
        int ix = rbCtx.findVirtualColumnNum(VirtualColumn.ROWISDELETED);
        value.cols[ix] = rowIsDeletedVector;
      }

      progress = baseReader.getProgress();
      return true;
    }
    // Once we have read the VectorizedRowBatchBase from the file, there are two kinds of cases
    // for which we might have to discard rows from the batch:
    // Case 1- when the row is created by a transaction that is not valid, or
    // Case 2- when the row has been deleted.
    // We will go through the batch to discover rows which match any of the cases and specifically
    // remove them from the selected vector. Of course, selectedInUse should also be set.

    BitSet selectedBitSet = new BitSet(vectorizedRowBatchBase.size);
    if (vectorizedRowBatchBase.selectedInUse) {
      // When selectedInUse is true, start with every bit set to false and selectively set
      // certain bits to true based on the selected[] vector.
      selectedBitSet.set(0, vectorizedRowBatchBase.size, false);
      for (int j = 0; j < vectorizedRowBatchBase.size; ++j) {
        int i = vectorizedRowBatchBase.selected[j];
        selectedBitSet.set(i);
      }
    } else {
      // When selectedInUse is set to false, everything in the batch is selected.
      selectedBitSet.set(0, vectorizedRowBatchBase.size, true);
    }
    ColumnVector[] innerRecordIdColumnVector = vectorizedRowBatchBase.cols;
    if (isOriginal) {
      // Handle synthetic row IDs for the original files.
      innerRecordIdColumnVector = handleOriginalFile(selectedBitSet, innerRecordIdColumnVector);
    } else {
      // Case 1- find rows which belong to write Ids that are not valid.
      findRecordsWithInvalidWriteIds(vectorizedRowBatchBase, selectedBitSet);
    }

    // Case 2- find rows which have been deleted.
    // if deleted rows should be fetched we clone the selectedBitSet to notDeletedBitSet.
    // Records marked by selectedBitSet should be filtered out from the result but notDeletedBitSet
    // should be appear with ROW__IS__DELETED = false
    BitSet notDeletedBitSet = fetchDeletedRows ? (BitSet) selectedBitSet.clone() : selectedBitSet;

    this.deleteEventRegistry.findDeletedRecords(innerRecordIdColumnVector,
        vectorizedRowBatchBase.size, notDeletedBitSet);

    if (selectedBitSet.cardinality() == vectorizedRowBatchBase.size) {
      // None of the cases above matched and everything is selected. Hence, we will use the
      // same values for the selected and selectedInUse.
      value.size = vectorizedRowBatchBase.size;
      value.selected = vectorizedRowBatchBase.selected;
      value.selectedInUse = vectorizedRowBatchBase.selectedInUse;
    } else {
      value.size = selectedBitSet.cardinality();
      value.selectedInUse = true;
      value.selected = new int[selectedBitSet.cardinality()];
      // This loop fills up the selected[] vector with all the index positions that are selected.
      for (int setBitIndex = selectedBitSet.nextSetBit(0), selectedItr = 0;
           setBitIndex >= 0;
           setBitIndex = selectedBitSet.nextSetBit(setBitIndex+1), ++selectedItr) {
        value.selected[selectedItr] = setBitIndex;
      }
    }

    copyFromBase(value);

    if (rowIdProjected) {
      int ix = rbCtx.findVirtualColumnNum(VirtualColumn.ROWID);
      value.cols[ix] = recordIdColumnVector;
    }
    if (rowIsDeletedProjected) {
      if (!fetchDeletedRows || notDeletedBitSet.cardinality() == vectorizedRowBatchBase.size) {
        rowIsDeletedVector.fill(false);
      } else {
        rowIsDeletedVector.set(notDeletedBitSet);
      }

      int ix = rbCtx.findVirtualColumnNum(VirtualColumn.ROWISDELETED);
      value.cols[ix] = rowIsDeletedVector;
    }
    progress = baseReader.getProgress();
    return true;
  }
There are 2 types of schema from the {@link #baseReader} that this handles.  In the case
the data was written to a transactional table from the start, every row is decorated with
transaction related info and looks like &lt;op, owid, writerId, rowid, cwid, &lt;f1, ... fn&gt;&gt;.

The other case is when data was written to non-transactional table and thus only has the user
data: &lt;f1, ... fn&gt;.  Then this table was then converted to a transactional table but the data
files are not changed until major compaction.  These are the "original" files.

In this case we may need to decorate the outgoing data with transactional column values at
read time.  (It's done somewhat out of band via VectorizedRowBatchCtx - ask Teddy Choi).
The "owid, writerId, rowid" columns represent {@link RecordIdentifier}.  They are assigned
each time the table is read in a way that needs to project {@link VirtualColumn#ROWID}.
Major compaction will attach these values to each row permanently.
It's critical that these generated column values are assigned exactly the same way by each
read of the same row and by the Compactor.
See {@link MRCompactor} and
{@link OrcRawRecordMerger.OriginalReaderPairToCompact} for the Compactor read path.
(Longer term should make compactor use this class)

This only decorates original rows with metadata if something above is requesting these values
or if there are Delete events to apply.

@return false where there is no more data, i.e. {@code value} is empty
ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcAcidRowBatchReader.java