Package | Description |
---|---|
org.apache.pig |
Public interfaces and classes for Pig.
|
org.apache.pig.backend.hadoop.executionengine | |
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer | |
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans | |
org.apache.pig.backend.hadoop.executionengine.physicalLayer |
Implementation of physical operators that use hadoop as the execution engine
and data storage.
|
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators | |
org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans | |
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators | |
org.apache.pig.backend.hadoop.executionengine.physicalLayer.util | |
org.apache.pig.backend.hadoop.executionengine.util | |
org.apache.pig.impl.util | |
org.apache.pig.newplan.logical.expression | |
org.apache.pig.newplan.logical.relational | |
org.apache.pig.pen |
Modifier and Type | Method and Description |
---|---|
protected PigStats |
PigServer.launchPlan(PhysicalPlan pp,
java.lang.String jobName)
A common method for launching the jobs according to the physical plan
|
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
HExecutionEngine.compile(LogicalPlan plan,
java.util.Properties properties) |
Modifier and Type | Field and Description |
---|---|
PhysicalPlan |
MapReduceOper.combinePlan |
PhysicalPlan |
MapReduceOper.mapPlan |
protected PhysicalPlan |
PigGenericMapBase.mp |
PhysicalPlan |
MapReduceOper.reducePlan |
protected PhysicalPlan |
PigGenericMapReduce.Reduce.rp |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
MRCompiler.getPlan()
Used to get the plan that was compiled
|
Modifier and Type | Method and Description |
---|---|
MROperPlan |
MapReduceLauncher.compile(PhysicalPlan php,
PigContext pc) |
void |
MapReduceLauncher.explain(PhysicalPlan php,
PigContext pc,
java.io.PrintStream ps,
java.lang.String format,
boolean verbose) |
abstract void |
Launcher.explain(PhysicalPlan pp,
PigContext pc,
java.io.PrintStream ps,
java.lang.String format,
boolean verbose)
Explain how a pig job will be executed on the underlying
infrastructure.
|
PigStats |
MapReduceLauncher.launchPig(PhysicalPlan php,
java.lang.String grpName,
PigContext pc) |
abstract PigStats |
Launcher.launchPig(PhysicalPlan php,
java.lang.String grpName,
PigContext pc)
Method to launch pig for hadoop either for a cluster's
job tracker or for a local job runner.
|
void |
PigGenericMapBase.setMapPlan(PhysicalPlan plan)
for local map/reduce simulation
|
void |
PigGenericMapReduce.Reduce.setReducePlan(PhysicalPlan plan)
Set the reduce plan: to be used by local runner for illustrator
|
Constructor and Description |
---|
MRCompiler(PhysicalPlan plan) |
MRCompiler(PhysicalPlan plan,
PigContext pigContext) |
PhyPlanSetter(PhysicalPlan plan) |
UDFFinishVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Constructor and Description |
---|
UDFFinishVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
DotMRPrinter.InnerOperator.getPlan() |
Modifier and Type | Method and Description |
---|---|
void |
UDFFinder.setPlan(PhysicalPlan plan) |
Constructor and Description |
---|
DotMRPrinter.InnerOperator(PhysicalPlan plan,
java.lang.String name) |
DotMRPrinter.InnerPlan(PhysicalPlan mapPlan,
PhysicalPlan combinePlan,
PhysicalPlan reducePlan) |
ScalarPhyFinder(PhysicalPlan plan) |
UDFFinder(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Constructor and Description |
---|
UDFFinder(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Field and Description |
---|---|
protected PhysicalPlan |
PhysicalOperator.parentPlan |
Modifier and Type | Method and Description |
---|---|
void |
PhysicalOperator.setParentPlan(PhysicalPlan physicalPlan) |
Constructor and Description |
---|
POPrinter(java.io.PrintStream ps,
PhysicalPlan pp) |
Modifier and Type | Method and Description |
---|---|
void |
UnaryExpressionOperator.setInputAsExpr(PhysicalPlan plan)
Set the contained expression to the be the input value.
|
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
PhysicalPlan.clone() |
Modifier and Type | Method and Description |
---|---|
protected java.util.Collection<PhysicalPlan> |
DotPOPrinter.getMultiOutputNestedPlans(PhysicalOperator op) |
protected java.util.Collection<PhysicalPlan> |
DotPOPrinter.getNestedPlans(PhysicalOperator op) |
Modifier and Type | Method and Description |
---|---|
protected DotPlanDumper |
DotPOPrinter.makeDumper(PhysicalPlan plan,
java.io.PrintStream ps) |
Constructor and Description |
---|
DotPOPrinter(PhysicalPlan plan,
java.io.PrintStream ps) |
DotPOPrinter(PhysicalPlan plan,
java.io.PrintStream ps,
boolean isSubGraph,
java.util.Set<Operator> subgraphs,
java.util.Set<Operator> multiInputSubgraphs,
java.util.Set<Operator> multiOutputSubgraphs) |
PhyPlanVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
XMLPhysicalPlanPrinter(PhysicalPlan plan,
org.w3c.dom.Document doc,
org.w3c.dom.Element parent) |
Constructor and Description |
---|
PhyPlanVisitor(PhysicalPlan plan,
PlanWalker<PhysicalOperator,PhysicalPlan> walker) |
Modifier and Type | Field and Description |
---|---|
protected java.util.List<PhysicalPlan> |
POForEach.inputPlans |
protected java.util.List<PhysicalPlan> |
POPreCombinerLocalRearrange.plans |
protected java.util.List<PhysicalPlan> |
POLocalRearrange.plans |
protected java.util.List<PhysicalPlan> |
POCollectedGroup.plans |
protected java.util.List<PhysicalPlan> |
POLocalRearrange.secondaryPlans |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
POPartialAgg.getKeyPlan() |
PhysicalPlan |
POLimit.getLimitPlan() |
PhysicalPlan |
POFilter.getPlan() |
Modifier and Type | Method and Description |
---|---|
java.util.List<PhysicalPlan> |
POCounter.getCounterPlans() |
java.util.List<PhysicalPlan> |
POMergeJoin.getInnerPlansOf(int index) |
java.util.List<PhysicalPlan> |
POJoinPackage.getInputPlans() |
java.util.List<PhysicalPlan> |
POForEach.getInputPlans() |
MultiMap<PhysicalOperator,PhysicalPlan> |
POSkewedJoin.getJoinPlans() |
java.util.List<java.util.List<PhysicalPlan>> |
POFRJoin.getJoinPlans() |
java.util.List<PhysicalPlan> |
POMergeCogroup.getLRInnerPlansOf(int i) |
java.util.List<PhysicalPlan> |
POSplit.getPlans()
Returns the list of nested plans.
|
java.util.List<PhysicalPlan> |
POPreCombinerLocalRearrange.getPlans() |
java.util.List<PhysicalPlan> |
POLocalRearrange.getPlans() |
java.util.List<PhysicalPlan> |
PODemux.getPlans()
Returns the list of inner plans.
|
java.util.List<PhysicalPlan> |
POCollectedGroup.getPlans() |
java.util.List<PhysicalPlan> |
PORank.getRankPlans() |
java.util.List<PhysicalPlan> |
POSort.getSortPlans() |
java.util.List<PhysicalPlan> |
POPartialAgg.getValuePlans() |
Modifier and Type | Method and Description |
---|---|
void |
POForEach.addInputPlan(PhysicalPlan plan,
boolean flatten) |
void |
POSplit.addPlan(PhysicalPlan inPlan)
Appends the specified plan to the end of
the nested input plan list
|
void |
PODemux.addPlan(PhysicalPlan inPlan)
Appends the specified plan at the end of the list.
|
void |
POSplit.removePlan(PhysicalPlan plan)
Removes plan from
the nested input plan list
|
void |
POPartialAgg.setKeyPlan(PhysicalPlan keyPlan) |
void |
POLimit.setLimitPlan(PhysicalPlan expressionPlan) |
void |
POFilter.setPlan(PhysicalPlan plan) |
void |
POMergeJoin.setupRightPipeline(PhysicalPlan rightPipeline) |
Modifier and Type | Method and Description |
---|---|
void |
POCounter.setCounterPlans(java.util.List<PhysicalPlan> counterPlans) |
void |
POJoinPackage.setInputPlans(java.util.List<PhysicalPlan> plans) |
void |
POForEach.setInputPlans(java.util.List<PhysicalPlan> plans) |
void |
POSkewedJoin.setJoinPlans(MultiMap<PhysicalOperator,PhysicalPlan> joinPlans) |
void |
POPreCombinerLocalRearrange.setPlans(java.util.List<PhysicalPlan> plans) |
void |
POLocalRearrange.setPlans(java.util.List<PhysicalPlan> plans) |
void |
POCollectedGroup.setPlans(java.util.List<PhysicalPlan> plans) |
void |
POLocalRearrange.setPlansFromCombiner(java.util.List<PhysicalPlan> plans) |
void |
PORank.setRankPlans(java.util.List<PhysicalPlan> rankPlans) |
void |
POLocalRearrange.setSecondaryPlans(java.util.List<PhysicalPlan> plans) |
void |
POSort.setSortPlans(java.util.List<PhysicalPlan> sortPlans) |
void |
POPartialAgg.setValuePlans(java.util.List<PhysicalPlan> valuePlans) |
Constructor and Description |
---|
POCounter(OperatorKey operatorKey,
int requestedParallelism,
java.util.List inp,
java.util.List<PhysicalPlan> counterPlans,
java.util.List<java.lang.Boolean> ascendingCol) |
POForEach(OperatorKey k,
int rp,
java.util.List<PhysicalPlan> inp,
java.util.List<java.lang.Boolean> isToBeFlattened) |
POForEach(OperatorKey operatorKey,
int requestedParallelism,
java.util.List<PhysicalPlan> innerPlans,
java.util.List<java.lang.Boolean> flattenList,
Schema schema) |
POFRJoin(OperatorKey k,
int rp,
java.util.List<PhysicalOperator> inp,
java.util.List<java.util.List<PhysicalPlan>> ppLists,
java.util.List<java.util.List<java.lang.Byte>> keyTypes,
FileSpec[] replFiles,
int fragment,
boolean isLeftOuter,
Tuple nullTuple) |
POFRJoin(OperatorKey k,
int rp,
java.util.List<PhysicalOperator> inp,
java.util.List<java.util.List<PhysicalPlan>> ppLists,
java.util.List<java.util.List<java.lang.Byte>> keyTypes,
FileSpec[] replFiles,
int fragment,
boolean isLeftOuter,
Tuple nullTuple,
Schema[] inputSchemas,
Schema[] keySchemas) |
POMergeJoin(OperatorKey k,
int rp,
java.util.List<PhysicalOperator> inp,
MultiMap<PhysicalOperator,PhysicalPlan> inpPlans,
java.util.List<java.util.List<java.lang.Byte>> keyTypes,
LOJoin.JOINTYPE joinType,
Schema leftInputSchema,
Schema rightInputSchema,
Schema mergedInputSchema) |
POOptimizedForEach(OperatorKey k,
int rp,
java.util.List<PhysicalPlan> inp,
java.util.List<java.lang.Boolean> isToBeFlattened) |
PORank(OperatorKey operatorKey,
int requestedParallelism,
java.util.List inp,
java.util.List<PhysicalPlan> rankPlans,
java.util.List<java.lang.Boolean> ascendingCol) |
POSort(OperatorKey k,
int rp,
java.util.List inp,
java.util.List<PhysicalPlan> sortPlans,
java.util.List<java.lang.Boolean> mAscCols,
POUserComparisonFunc mSortFunc) |
Modifier and Type | Method and Description |
---|---|
static <C extends PhysicalOperator> |
PlanHelper.containsPhysicalOperator(PhysicalPlan plan,
java.lang.Class<C> opClass) |
static <C extends PhysicalOperator> |
PlanHelper.getPhysicalOperators(PhysicalPlan plan,
java.lang.Class<C> opClass)
Returns a LinkedList of operators contained within the physical plan which implement the supplied class, in dependency order.
|
Modifier and Type | Method and Description |
---|---|
static FileSpec |
MapRedUtil.checkLeafIsStore(PhysicalPlan plan,
PigContext pigContext) |
Modifier and Type | Method and Description |
---|---|
static void |
CompilerUtils.addEmptyBagOuterJoin(PhysicalPlan fePlan,
Schema inputSchema) |
Modifier and Type | Field and Description |
---|---|
protected PhysicalPlan |
ExpToPhyTranslationVisitor.currentPlan |
Modifier and Type | Field and Description |
---|---|
protected java.util.Deque<PhysicalPlan> |
ExpToPhyTranslationVisitor.currentPlans |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
ExpToPhyTranslationVisitor.getPhysicalPlan() |
Constructor and Description |
---|
ExpToPhyTranslationVisitor(OperatorPlan plan,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
java.util.Map<Operator,PhysicalOperator> map) |
ExpToPhyTranslationVisitor(OperatorPlan plan,
PlanWalker walker,
LogicalRelationalOperator op,
PhysicalPlan phyPlan,
java.util.Map<Operator,PhysicalOperator> map) |
Modifier and Type | Field and Description |
---|---|
protected PhysicalPlan |
LogToPhyTranslationVisitor.currentPlan |
Modifier and Type | Field and Description |
---|---|
protected java.util.Deque<PhysicalPlan> |
LogToPhyTranslationVisitor.currentPlans |
Modifier and Type | Method and Description |
---|---|
PhysicalPlan |
LogToPhyTranslationVisitor.getPhysicalPlan() |
Modifier and Type | Method and Description |
---|---|
static void |
LogToPhyTranslationVisitor.updateWithEmptyBagCheck(PhysicalPlan fePlan,
Operator joinInput)
updates plan with check for empty bag and if bag is empty to flatten a bag
with as many null's as dictated by the schema
|
Modifier and Type | Method and Description |
---|---|
static java.util.Map<LogicalRelationalOperator,java.util.Collection<IdentityHashSet<Tuple>>> |
EquivalenceClasses.getLoToEqClassMap(PhysicalPlan plan,
LogicalPlan lp,
java.util.Map<Operator,PhysicalOperator> logToPhyMap,
java.util.Map<Operator,DataBag> logToDataMap,
java.util.Map<LOForEach,java.util.Map<LogicalRelationalOperator,PhysicalOperator>> forEachInnerLogToPhyMap,
java.util.HashMap<PhysicalOperator,java.util.Collection<IdentityHashSet<Tuple>>> poToEqclassesMap) |
void |
LocalMapReduceSimulator.launchPig(PhysicalPlan php,
java.util.Map<LOLoad,DataBag> baseData,
LineageTracer lineage,
IllustratorAttacher attacher,
ExampleGenerator eg,
PigContext pc) |
void |
IllustratorAttacher.revisit(PhysicalPlan plan)
revisit an enhanced physical plan from MR compilation
|
Constructor and Description |
---|
IllustratorAttacher(PhysicalPlan plan,
LineageTracer lineage,
int maxRecords,
java.util.Map<POLoad,LogicalSchema> poLoadToSchemaMap,
PigContext hadoopPigContext) |
LineageTrimmingVisitor(LogicalPlan plan,
java.util.Map<LOLoad,DataBag> baseData,
ExampleGenerator eg,
java.util.Map<Operator,PhysicalOperator> LogToPhyMap,
PhysicalPlan physPlan,
PigContext pc) |
PhysicalPlanResetter(PhysicalPlan plan) |
Copyright © 2007-2012 The Apache Software Foundation