@SuppressWarnings("nls")
  private Operator genGroupByPlanMapAggr2MR(String dest, QB qb,
                                            Operator inputOperatorInfo) throws SemanticException {

    QBParseInfo parseInfo = qb.getParseInfo();

    Pair<List<ASTNode>, List<Long>> grpByExprsGroupingSets = getGroupByGroupingSetsForClause(parseInfo, dest);

    List<ASTNode> grpByExprs = grpByExprsGroupingSets.getLeft();
    List<Long> groupingSets = grpByExprsGroupingSets.getRight();
    boolean groupingSetsPresent = !groupingSets.isEmpty();

    if (groupingSetsPresent) {

      int newMRJobGroupingSetsThreshold =
          conf.getIntVar(HiveConf.ConfVars.HIVE_NEW_JOB_GROUPING_SET_CARDINALITY);

      // Turn off skew if an additional MR job is required anyway for grouping sets.
      if (groupingSets.size() > newMRJobGroupingSetsThreshold) {
        String errorMsg = "The number of rows per input row due to grouping sets is "
            + groupingSets.size();
        throw new SemanticException(
            ErrorMsg.HIVE_GROUPING_SETS_THRESHOLD_NOT_ALLOWED_WITH_SKEW.getMsg(errorMsg));
      }
    }

    // ////// Generate GroupbyOperator for a map-side partial aggregation
    Map<String, GenericUDAFEvaluator> genericUDAFEvaluators =
        new LinkedHashMap<String, GenericUDAFEvaluator>();
    GroupByOperator groupByOperatorInfo =
        (GroupByOperator) genGroupByPlanMapGroupByOperator(
            qb, dest, grpByExprs, inputOperatorInfo,
            genericUDAFEvaluators, groupingSets, groupingSetsPresent);

    groupOpToInputTables.put(groupByOperatorInfo, opParseCtx.get(
        inputOperatorInfo).getRowResolver().getTableNames());
    // Optimize the scenario when there are no grouping keys and no distinct - 2
    // map-reduce jobs are not needed
    // For eg: select count(1) from T where t.ds = ....
    if (!optimizeMapAggrGroupBy(dest, qb)) {
      List<ASTNode> distinctFuncExprs = parseInfo.getDistinctFuncExprsForClause(dest);

      // ////// Generate ReduceSink Operator
      Operator reduceSinkOperatorInfo =
          genGroupByPlanReduceSinkOperator(qb,
              dest,
              groupByOperatorInfo,
              grpByExprs,
              distinctFuncExprs.isEmpty() ? -1 : Integer.MAX_VALUE,
              false,
              -1,
              true,
              groupingSetsPresent);

      // ////// Generate GroupbyOperator for a partial aggregation
      Operator groupByOperatorInfo2 = genGroupByPlanGroupByOperator1(parseInfo,
          dest, reduceSinkOperatorInfo, GroupByDesc.Mode.PARTIALS,
          genericUDAFEvaluators,
          groupingSets, groupingSetsPresent, false);

      int numReducers = -1;
      if (grpByExprs.isEmpty()) {
        numReducers = 1;
      }

      // ////// Generate ReduceSinkOperator2
      Operator reduceSinkOperatorInfo2 = genGroupByPlanReduceSinkOperator2MR(
          parseInfo, dest, groupByOperatorInfo2, grpByExprs.size(), numReducers,
          groupingSetsPresent);

      // ////// Generate GroupbyOperator3
      return genGroupByPlanGroupByOperator2MR(parseInfo, dest,
          reduceSinkOperatorInfo2,
          genericUDAFEvaluators, groupingSetsPresent);
    } else {
      // If there are no grouping keys, grouping sets cannot be present
      assert !groupingSetsPresent;

      // ////// Generate ReduceSink Operator
      Operator reduceSinkOperatorInfo =
          genGroupByPlanReduceSinkOperator(qb,
              dest,
              groupByOperatorInfo,
              grpByExprs,
              grpByExprs.size(),
              false,
              1,
              true,
              groupingSetsPresent);

      return genGroupByPlanGroupByOperator2MR(parseInfo, dest,
          reduceSinkOperatorInfo, genericUDAFEvaluators, false);
    }
  }
Generate a Group-By plan using a 2 map-reduce jobs. However, only 1
group-by plan is generated if the query involves no grouping key and no
distincts. In that case, the plan is same as generated by
genGroupByPlanMapAggr1MR. Otherwise, the following plan is generated: First
perform a map side partial aggregation (to reduce the amount of data). Then
spray by the grouping key and distinct key (or a random number, if no
distinct is present) in hope of getting a uniform distribution, and compute
partial aggregates grouped by the reduction key (grouping key + distinct
key). Evaluate partial aggregates first, and spray by the grouping key to
compute actual aggregates in the second phase.

The aggregation evaluation functions are as follows:

No grouping sets:
STAGE 1
Group by Operator:
grouping keys: group by expressions if no DISTINCT
grouping keys: group by expressions + distinct keys if DISTINCT
Mapper: iterate/terminatePartial (mode = HASH)
Partitioning Key: random() if no DISTINCT
grouping + distinct key if DISTINCT
Sorting Key: grouping key if no DISTINCT
grouping + distinct key if DISTINCT
Reducer: iterate/terminatePartial if DISTINCT
merge/terminatePartial if NO DISTINCT (mode = MERGEPARTIAL)
Group by Operator:
grouping keys: group by expressions

STAGE 2
Partitioning Key: grouping key
Sorting Key: grouping key
Reducer: merge/terminate (mode = FINAL)

In the presence of grouping sets, the aggregation evaluation functions are as follows:
STAGE 1
Group by Operator:
grouping keys: group by expressions + grouping id. if no DISTINCT
grouping keys: group by expressions + + grouping id. + distinct keys if DISTINCT
Mapper: iterate/terminatePartial (mode = HASH)
Partitioning Key: random() if no DISTINCT
grouping + grouping id. + distinct key if DISTINCT
Sorting Key: grouping key + grouping id. if no DISTINCT
grouping + grouping id. + distinct key if DISTINCT
Reducer: iterate/terminatePartial if DISTINCT
merge/terminatePartial if NO DISTINCT (mode = MERGEPARTIAL)
Group by Operator:
grouping keys: group by expressions + grouping id.

STAGE 2
Partitioning Key: grouping key
Sorting Key: grouping key + grouping id.
Reducer: merge/terminate (mode = FINAL)
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java