Subzero: Use CFG-local arena allocation for relevant containers.

ALLOCATION.rst

+Object allocation and lifetime in ICE
+=====================================
+
+This document discusses object lifetime and scoping issues, starting with
+bitcode parsing and ending with ELF file emission.
+
+Multithreaded translation model
+-------------------------------
+
+A single thread is responsible for parsing PNaCl bitcode (possibly concurrently
+with downloading the bitcode file) and constructing the initial high-level ICE.
+The result is a queue of Cfg pointers.  The parser thread incrementally adds a
+Cfg pointer to the queue after the Cfg is created, and then moves on to parse
+the next function.
+
+Multiple translation worker threads draw from the queue of Cfg pointers as they
+are added to the queue, such that several functions can be translated in parallel.
+The result is a queue of assembler buffers, each of which consists of machine code
+plus fixups.
+
+A single thread is responsible for writing the assembler buffers to an ELF file.
+It consumes the assembler buffers from the queue that the translation threads
+write to.
+
+This means that Cfgs are created by the parser thread and destroyed by the
+translation thread (including Cfg nodes, instructions, and most kinds of
+operands), and assembler buffers are created by the translation thread and
+destroyed by the writer thread.
+
+Deterministic execution
+^^^^^^^^^^^^^^^^^^^^^^^
+
+Although code randomization is a key aspect of security, deterministic and
+repeatable translation is sometimes needed, e.g. for regression testing.
+Multithreaded translation introduces potential for randomness that may need to
+be made deterministic.
+
+* Bitcode parsing is sequential, so it's easy to use a FIFO queue to keep the
+  translation queue in deterministic order.  But since translation is
+  multithreaded, FIFO order for the assembler buffer queue may not be
+  deterministic.  The writer thread would be responsible for reordering the
+  buffers, potentially waiting for slower translations to complete even if other
+  assembler buffers are available.
+
+* Different translation threads may add new constant pool entries at different
+  times.  Some constant pool entries are emitted as read-only data.  This
+  includes floating-point constants for x86, as well as integer immediate
+  randomization through constant pooling.  These constant pool entries are
+  emitted after all assembler buffers have been written.  The writer needs to be
+  able to sort them deterministically before emitting them.
+
+Object lifetimes
+----------------
+
+Objects of type Constant, or a subclass of Constant, are pooled globally.  The
+pooling is managed by the GlobalContext class.  Since Constants are added or
+looked up by translation threads and the parser thread, access to the constant
+pools, as well as GlobalContext in general, need to be arbitrated by locks.
+(It's possible that if there's too much contention, we can maintain a
+thread-local cache for Constant pool lookups.)  Constants live across all
+function translations, and are destroyed only at the end.
+
+Several object types are scoped within the lifetime of the Cfg.  These include
+CfgNode, Inst, Variable, and any target-specific subclasses of Inst and Operand.
+When the Cfg is destroyed, these scoped objects are destroyed as well.  To keep
+this cheap, the Cfg includes a slab allocator from which these objects are
+allocated, and the objects should not contain fields with non-trivial
+destructors.  Most of these fields are POD, but in a couple of cases these
+fields are STL containers.  We deal with this, and avoid leaking memory, by
+providing the container with an allocator that uses the Cfg-local slab
+allocator.  Since the container allocator generally needs to be stateless, we
+store a pointer to the slab allocator in thread-local storage (TLS).  This is
+straightforward since on any of the threads, only one Cfg is active at a time,
+and a given Cfg is only active in one thread at a time (either the parser
+thread, or at most one translation thread, or the writer thread).
+
+Even though there is a one-to-one correspondence between Cfgs and assembler
+buffers, they need to use different allocators.  This is because the translation
+thread wants to destroy the Cfg and reclaim all its memory after translation
+completes, but possibly before the assembly buffer is written to the ELF file.
+Ownership of the assembler buffer and its allocator are transferred to the
+writer thread after translation completes, similar to the way ownership of the
+Cfg and its allocator are transferred to the translation thread after parsing
+completes.
+
+Allocators and TLS
+------------------
+
+Part of the Cfg building, and transformations on the Cfg, include STL container
+operations which may need to allocate additional memory in a stateless fashion.
+This requires maintaining the proper slab allocator pointer in TLS.
+
+When the parser thread creates a new Cfg object, it puts a pointer to the Cfg's
+slab allocator into its own TLS.  This is used as the Cfg is built within the
+parser thread.  After the Cfg is built, the parser thread clears its allocator
+pointer, adds the new Cfg pointer to the translation queue, continues with the
+next function.
+
+When the translation thread grabs a new Cfg pointer, it installs the Cfg's slab
+allocator into its TLS and translates the function.  When generating the
+assembly buffer, it must take care not to use the Cfg's slab allocator.  If
+there is a slab allocator for the assembler buffer, a pointer to it can also be
+installed in TLS if needed.
+
+The translation thread destroys the Cfg when it is done translating, including
+the Cfg's slab allocator, and clears the allocator pointer from its TLS.
+Likewise, the writer thread destroys the assembler buffer when it is finished
+with it.
+
+Thread safety
+-------------
+
+The parse/translate/write stages of the translation pipeline are fairly
+independent, with little opportunity for threads to interfere.  The Subzero
+design calls for all shared accesses to go through the GlobalContext, which adds
+locking as appropriate.  This includes the coarse-grain work queues for Cfgs and
+assembler buffers.  It also includes finer-grain access to constant pool
+entries, as well as output streams for verbose debugging output.
+
+If locked access to constant pools becomes a bottleneck, we can investigate
+thread-local caches of constants (as mentioned earlier).  Also, it should be
+safe though slightly less efficient to allow duplicate copies of constants
+across threads (which could be de-dupped by the writer at the end).
+
+We will use ThreadSanitizer as a way to detect potential data races in the
+implementation.

src/IceCfg.cpp

@@ -23,11 +23,17 @@
 
 namespace Ice {
 
+thread_local const Cfg *Cfg::CurrentCfg = NULL;
+
+ArenaAllocator *getCurrentCfgAllocator() {
+  return Cfg::getCurrentCfgAllocator();
+}
+
 Cfg::Cfg(GlobalContext *Ctx)
     : Ctx(Ctx), FunctionName(""), ReturnType(IceType_void),
       IsInternalLinkage(false), HasError(false), FocusedTiming(false),
       ErrorMessage(""), Entry(NULL), NextInstNumber(Inst::NumberInitial),
-      Live(nullptr),
+      Allocator(new ArenaAllocator()), Live(nullptr),
       Target(TargetLowering::createLowering(Ctx->getTargetArch(), this)),
       VMetadata(new VariablesMetadata(this)),
       TargetAssembler(
@@ -37,7 +43,13 @@ Cfg::Cfg(GlobalContext *Ctx)
          "Attempt to build cfg when IR generation disabled");
 }
 
-Cfg::~Cfg() {}
+Cfg::~Cfg() {
+  // TODO(stichnot,kschimpf): Set CurrentCfg=NULL in the dtor for
+  // safety.  This can't be done currently because the translator
+  // manages the Cfg by creating a new Cfg (which sets CurrentCfg to
+  // the new value), then deleting the old Cfg (which would then reset
+  // CurrentCfg to NULL).
+}
 
 void Cfg::setError(const IceString &Message) {
   HasError = true;

src/IceCfg.h

@@ -17,8 +17,6 @@
 
 #include <memory>
 
-#include "llvm/Support/Allocator.h"
-
 #include "assembler.h"
 #include "IceClFlags.h"
 #include "IceDefs.h"
@@ -32,9 +30,24 @@ class Cfg {
   Cfg &operator=(const Cfg &) = delete;
 
 public:
-  Cfg(GlobalContext *Ctx);
   ~Cfg();
 
+  // TODO(stichnot): Change this to return unique_ptr<Cfg>, and plumb
+  // it through the callers, to make ownership and lifetime and
+  // destruction requirements more explicit.
+  static Cfg *create(GlobalContext *Ctx) {
+    Cfg *Func = new Cfg(Ctx);
+    CurrentCfg = Func;
+    return Func;
+  }
+  // Gets a pointer to the current thread's Cfg.
+  static const Cfg *getCurrentCfg() { return CurrentCfg; }
+  // Gets a pointer to the current thread's Cfg's allocator.
+  static ArenaAllocator *getCurrentCfgAllocator() {
+    assert(CurrentCfg);
+    return CurrentCfg->Allocator.get();
+  }
+
   GlobalContext *getContext() const { return Ctx; }
 
   // Manage the name and return type of the function being translated.
@@ -150,38 +163,25 @@ public:
   void dump(const IceString &Message = "");
 
   // Allocate data of type T using the per-Cfg allocator.
-  template <typename T> T *allocate() { return Allocator.Allocate<T>(); }
-
-  // Allocate an instruction of type T using the per-Cfg instruction allocator.
-  template <typename T> T *allocateInst() { return Allocator.Allocate<T>(); }
+  template <typename T> T *allocate() { return Allocator->Allocate<T>(); }
 
   // Allocate an array of data of type T using the per-Cfg allocator.
   template <typename T> T *allocateArrayOf(size_t NumElems) {
-    return Allocator.Allocate<T>(NumElems);
+    return Allocator->Allocate<T>(NumElems);
   }
 
   // Deallocate data that was allocated via allocate<T>().
   template <typename T> void deallocate(T *Object) {
-    Allocator.Deallocate(Object);
-  }
-
-  // Deallocate data that was allocated via allocateInst<T>().
-  template <typename T> void deallocateInst(T *Instr) {
-    Allocator.Deallocate(Instr);
+    Allocator->Deallocate(Object);
   }
 
   // Deallocate data that was allocated via allocateArrayOf<T>().
   template <typename T> void deallocateArrayOf(T *Array) {
-    Allocator.Deallocate(Array);
+    Allocator->Deallocate(Array);
   }
 
 private:
-  // TODO: for now, everything is allocated from the same allocator. In the
-  // future we may want to split this to several allocators, for example in
-  // order to use a "Recycler" to preserve memory. If we keep all allocation
-  // requests from the Cfg exposed via methods, we can always switch the
-  // implementation over at a later point.
-  llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 1024 * 1024> Allocator;
+  Cfg(GlobalContext *Ctx);
 
   GlobalContext *Ctx;
   IceString FunctionName;
@@ -197,6 +197,7 @@ private:
   VarList Variables;
   VarList Args; // subset of Variables, in argument order
   VarList ImplicitArgs; // subset of Variables
+  std::unique_ptr<ArenaAllocator> Allocator;
   std::unique_ptr<Liveness> Live;
   std::unique_ptr<TargetLowering> Target;
   std::unique_ptr<VariablesMetadata> VMetadata;
@@ -208,6 +209,11 @@ private:
   // register allocation, resetCurrentNode() should be called to avoid
   // spurious validation failures.
   const CfgNode *CurrentNode;
+
+  // Maintain a pointer in TLS to the current Cfg being translated.
+  // This is primarily for accessing its allocator statelessly, but
+  // other uses are possible.
+  thread_local static const Cfg *CurrentCfg;
 };
 
 } // end of namespace Ice

src/IceConverter.cpp

@@ -83,7 +83,7 @@ public:
   Ice::Cfg *convertFunction(const Function *F) {
     VarMap.clear();
     NodeMap.clear();
-    Func = new Ice::Cfg(Ctx);
+    Func = Ice::Cfg::create(Ctx);
     Func->setFunctionName(F->getName());
     Func->setReturnType(convertToIceType(F->getReturnType()));
     Func->setInternal(F->hasInternalLinkage());

src/IceDefs.h

@@ -32,6 +32,7 @@
 #include "llvm/ADT/SmallBitVector.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Allocator.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/ELF.h"
 #include "llvm/Support/raw_ostream.h"
@@ -56,16 +57,40 @@ class Variable;
 class VariableDeclaration;
 class VariablesMetadata;
 
-// TODO: Switch over to LLVM's ADT container classes.
-// http://llvm.org/docs/ProgrammersManual.html#picking-the-right-data-structure-for-a-task
+typedef llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 1024 * 1024>
+ArenaAllocator;
+
+ArenaAllocator *getCurrentCfgAllocator();
+
+template <typename T> struct CfgLocalAllocator {
+  using value_type = T;
+  CfgLocalAllocator() = default;
+  template <class U> CfgLocalAllocator(const CfgLocalAllocator<U> &) {}
+  T *allocate(std::size_t Num) {
+    return getCurrentCfgAllocator()->Allocate<T>(Num);
+  }
+  void deallocate(T *, std::size_t) {}
+};
+template <typename T, typename U>
+inline bool operator==(const CfgLocalAllocator<T> &,
+                       const CfgLocalAllocator<U> &) {
+  return true;
+}
+template <typename T, typename U>
+inline bool operator!=(const CfgLocalAllocator<T> &,
+                       const CfgLocalAllocator<U> &) {
+  return false;
+}
+
 typedef std::string IceString;
 typedef llvm::ilist<Inst> InstList;
 // Ideally PhiList would be llvm::ilist<InstPhi>, and similar for
 // AssignList, but this runs into issues with SFINAE.
 typedef InstList PhiList;
 typedef InstList AssignList;
-typedef std::vector<Variable *> VarList;
-typedef std::vector<CfgNode *> NodeList;
+// VarList and NodeList are arena-allocated from the Cfg's allocator.
+typedef std::vector<Variable *, CfgLocalAllocator<Variable *>> VarList;
+typedef std::vector<CfgNode *, CfgLocalAllocator<CfgNode *>> NodeList;
 typedef std::vector<Constant *> ConstantList;
 
 // SizeT is for holding small-ish limits like number of source

src/IceGlobalContext.h

@@ -17,8 +17,6 @@
 
 #include <memory>
 
-#include "llvm/Support/Allocator.h"
-
 #include "IceDefs.h"
 #include "IceClFlags.h"
 #include "IceELFObjectWriter.h"
@@ -208,7 +206,7 @@ private:
   Ostream *StrDump; // Stream for dumping / diagnostics
   Ostream *StrEmit; // Stream for code emission
 
-  llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 1024 * 1024> Allocator;
+  ArenaAllocator Allocator;
   VerboseMask VMask;
   std::unique_ptr<class ConstantPool> ConstPool;
   Intrinsics IntrinsicsInfo;

src/IceInst.h

@@ -230,7 +230,7 @@ class InstAlloca : public InstHighLevel {
 public:
   static InstAlloca *create(Cfg *Func, Operand *ByteCount,
                             uint32_t AlignInBytes, Variable *Dest) {
-    return new (Func->allocateInst<InstAlloca>())
+    return new (Func->allocate<InstAlloca>())
         InstAlloca(Func, ByteCount, AlignInBytes, Dest);
   }
   uint32_t getAlignInBytes() const { return AlignInBytes; }
@@ -261,7 +261,7 @@ public:
 
   static InstArithmetic *create(Cfg *Func, OpKind Op, Variable *Dest,
                                 Operand *Source1, Operand *Source2) {
-    return new (Func->allocateInst<InstArithmetic>())
+    return new (Func->allocate<InstArithmetic>())
         InstArithmetic(Func, Op, Dest, Source1, Source2);
   }
   OpKind getOp() const { return Op; }
@@ -292,8 +292,7 @@ class InstAssign : public InstHighLevel {
 
 public:
   static InstAssign *create(Cfg *Func, Variable *Dest, Operand *Source) {
-    return new (Func->allocateInst<InstAssign>())
-        InstAssign(Func, Dest, Source);
+    return new (Func->allocate<InstAssign>()) InstAssign(Func, Dest, Source);
   }
   bool isSimpleAssign() const override { return true; }
   void dump(const Cfg *Func) const override;
@@ -315,12 +314,12 @@ public:
   // optimized to an unconditional branch.
   static InstBr *create(Cfg *Func, Operand *Source, CfgNode *TargetTrue,
                         CfgNode *TargetFalse) {
-    return new (Func->allocateInst<InstBr>())
+    return new (Func->allocate<InstBr>())
         InstBr(Func, Source, TargetTrue, TargetFalse);
   }
   // Create an unconditional branch.
   static InstBr *create(Cfg *Func, CfgNode *Target) {
-    return new (Func->allocateInst<InstBr>()) InstBr(Func, Target);
+    return new (Func->allocate<InstBr>()) InstBr(Func, Target);
   }
   bool isUnconditional() const { return getTargetTrue() == NULL; }
   Operand *getCondition() const {
@@ -364,7 +363,7 @@ public:
     // particular intrinsic is deletable and has no side-effects.
     const bool HasSideEffects = true;
     const InstKind Kind = Inst::Call;
-    return new (Func->allocateInst<InstCall>()) InstCall(
+    return new (Func->allocate<InstCall>()) InstCall(
         Func, NumArgs, Dest, CallTarget, HasTailCall, HasSideEffects, Kind);
   }
   void addArg(Operand *Arg) { addSource(Arg); }
@@ -406,7 +405,7 @@ public:
 
   static InstCast *create(Cfg *Func, OpKind CastKind, Variable *Dest,
                           Operand *Source) {
-    return new (Func->allocateInst<InstCast>())
+    return new (Func->allocate<InstCast>())
         InstCast(Func, CastKind, Dest, Source);
   }
   OpKind getCastKind() const { return CastKind; }
@@ -427,7 +426,7 @@ class InstExtractElement : public InstHighLevel {
 public:
   static InstExtractElement *create(Cfg *Func, Variable *Dest, Operand *Source1,
                                     Operand *Source2) {
-    return new (Func->allocateInst<InstExtractElement>())
+    return new (Func->allocate<InstExtractElement>())
         InstExtractElement(Func, Dest, Source1, Source2);
   }
 
@@ -458,7 +457,7 @@ public:
 
   static InstFcmp *create(Cfg *Func, FCond Condition, Variable *Dest,
                           Operand *Source1, Operand *Source2) {
-    return new (Func->allocateInst<InstFcmp>())
+    return new (Func->allocate<InstFcmp>())
         InstFcmp(Func, Condition, Dest, Source1, Source2);
   }
   FCond getCondition() const { return Condition; }
@@ -488,7 +487,7 @@ public:
 
   static InstIcmp *create(Cfg *Func, ICond Condition, Variable *Dest,
                           Operand *Source1, Operand *Source2) {
-    return new (Func->allocateInst<InstIcmp>())
+    return new (Func->allocate<InstIcmp>())
         InstIcmp(Func, Condition, Dest, Source1, Source2);
   }
   ICond getCondition() const { return Condition; }
@@ -510,7 +509,7 @@ class InstInsertElement : public InstHighLevel {
 public:
   static InstInsertElement *create(Cfg *Func, Variable *Dest, Operand *Source1,
                                    Operand *Source2, Operand *Source3) {
-    return new (Func->allocateInst<InstInsertElement>())
+    return new (Func->allocate<InstInsertElement>())
         InstInsertElement(Func, Dest, Source1, Source2, Source3);
   }
 
@@ -535,7 +534,7 @@ public:
   static InstIntrinsicCall *create(Cfg *Func, SizeT NumArgs, Variable *Dest,
                                    Operand *CallTarget,
                                    const Intrinsics::IntrinsicInfo &Info) {
-    return new (Func->allocateInst<InstIntrinsicCall>())
+    return new (Func->allocate<InstIntrinsicCall>())
         InstIntrinsicCall(Func, NumArgs, Dest, CallTarget, Info);
   }
   static bool classof(const Inst *Inst) {
@@ -564,8 +563,7 @@ public:
                           uint32_t Align = 1) {
     // TODO(kschimpf) Stop ignoring alignment specification.
     (void)Align;
-    return new (Func->allocateInst<InstLoad>())
-        InstLoad(Func, Dest, SourceAddr);
+    return new (Func->allocate<InstLoad>()) InstLoad(Func, Dest, SourceAddr);
   }
   Operand *getSourceAddress() const { return getSrc(0); }
   void dump(const Cfg *Func) const override;
@@ -584,7 +582,7 @@ class InstPhi : public InstHighLevel {
 
 public:
   static InstPhi *create(Cfg *Func, SizeT MaxSrcs, Variable *Dest) {
-    return new (Func->allocateInst<InstPhi>()) InstPhi(Func, MaxSrcs, Dest);
+    return new (Func->allocate<InstPhi>()) InstPhi(Func, MaxSrcs, Dest);
   }
   void addArgument(Operand *Source, CfgNode *Label);
   Operand *getOperandForTarget(CfgNode *Target) const;
@@ -618,7 +616,7 @@ class InstRet : public InstHighLevel {
 
 public:
   static InstRet *create(Cfg *Func, Operand *RetValue = NULL) {
-    return new (Func->allocateInst<InstRet>()) InstRet(Func, RetValue);
+    return new (Func->allocate<InstRet>()) InstRet(Func, RetValue);
   }
   bool hasRetValue() const { return getSrcSize(); }
   Operand *getRetValue() const {
@@ -642,7 +640,7 @@ class InstSelect : public InstHighLevel {
 public:
   static InstSelect *create(Cfg *Func, Variable *Dest, Operand *Condition,
                             Operand *SourceTrue, Operand *SourceFalse) {
-    return new (Func->allocateInst<InstSelect>())
+    return new (Func->allocate<InstSelect>())
         InstSelect(Func, Dest, Condition, SourceTrue, SourceFalse);
   }
   Operand *getCondition() const { return getSrc(0); }
@@ -668,7 +666,7 @@ public:
                            uint32_t align = 1) {
     // TODO(kschimpf) Stop ignoring alignment specification.
     (void)align;
-    return new (Func->allocateInst<InstStore>()) InstStore(Func, Data, Addr);
+    return new (Func->allocate<InstStore>()) InstStore(Func, Data, Addr);
   }
   Operand *getAddr() const { return getSrc(1); }
   Operand *getData() const { return getSrc(0); }
@@ -689,7 +687,7 @@ class InstSwitch : public InstHighLevel {
 public:
   static InstSwitch *create(Cfg *Func, SizeT NumCases, Operand *Source,
                             CfgNode *LabelDefault) {
-    return new (Func->allocateInst<InstSwitch>())
+    return new (Func->allocate<InstSwitch>())
         InstSwitch(Func, NumCases, Source, LabelDefault);
   }
   Operand *getComparison() const { return getSrc(0); }
@@ -732,7 +730,7 @@ class InstUnreachable : public InstHighLevel {
 
 public:
   static InstUnreachable *create(Cfg *Func) {
-    return new (Func->allocateInst<InstUnreachable>()) InstUnreachable(Func);
+    return new (Func->allocate<InstUnreachable>()) InstUnreachable(Func);
   }
   NodeList getTerminatorEdges() const override { return NodeList(); }
   void dump(const Cfg *Func) const override;
@@ -763,7 +761,7 @@ class InstFakeDef : public InstHighLevel {
 
 public:
   static InstFakeDef *create(Cfg *Func, Variable *Dest, Variable *Src = NULL) {
-    return new (Func->allocateInst<InstFakeDef>()) InstFakeDef(Func, Dest, Src);
+    return new (Func->allocate<InstFakeDef>()) InstFakeDef(Func, Dest, Src);
   }
   void emit(const Cfg *Func) const override;
   void emitIAS(const Cfg * /* Func */) const override {}
@@ -786,7 +784,7 @@ class InstFakeUse : public InstHighLevel {
 
 public:
   static InstFakeUse *create(Cfg *Func, Variable *Src) {
-    return new (Func->allocateInst<InstFakeUse>()) InstFakeUse(Func, Src);
+    return new (Func->allocate<InstFakeUse>()) InstFakeUse(Func, Src);
   }
   void emit(const Cfg *Func) const override;
   void emitIAS(const Cfg * /* Func */) const override {}
@@ -814,7 +812,7 @@ class InstFakeKill : public InstHighLevel {
 
 public:
   static InstFakeKill *create(Cfg *Func, const Inst *Linked) {
-    return new (Func->allocateInst<InstFakeKill>()) InstFakeKill(Func, Linked);
+    return new (Func->allocate<InstFakeKill>()) InstFakeKill(Func, Linked);
   }
   const Inst *getLinked() const { return Linked; }
   void emit(const Cfg *Func) const override;

src/IceInstX8632.cpp

@@ -2023,7 +2023,7 @@ template <> void InstX8632Lea::emit(const Cfg *Func) const {
     Type Ty = Src0Var->getType();
     // lea on x86-32 doesn't accept mem128 operands, so cast VSrc0 to an
     // acceptable type.
-    Src0Var->asType(isVectorType(Ty) ? IceType_i32 : Ty).emit(Func);
+    Src0Var->asType(isVectorType(Ty) ? IceType_i32 : Ty)->emit(Func);
   } else {
     Src0->emit(Func);
   }
@@ -2056,7 +2056,7 @@ template <> void InstX8632Mov::emit(const Cfg *Func) const {
          Func->getTarget()->typeWidthInBytesOnStack(SrcTy));
   Src->emit(Func);
   Str << ", ";
-  getDest()->asType(SrcTy).emit(Func);
+  getDest()->asType(SrcTy)->emit(Func);
 }
 
 template <> void InstX8632Mov::emitIAS(const Cfg *Func) const {
@@ -2446,7 +2446,7 @@ template <> void InstX8632Pextr::emit(const Cfg *Func) const {
   // a memory dest, but we aren't using it. For uniformity, just restrict
   // them all to have a register dest for now.
   assert(Dest->hasReg());
-  Dest->asType(IceType_i32).emit(Func);
+  Dest->asType(IceType_i32)->emit(Func);
 }
 
 template <> void InstX8632Pextr::emitIAS(const Cfg *Func) const {
@@ -2489,7 +2489,7 @@ template <> void InstX8632Pinsr::emit(const Cfg *Func) const {
   if (const auto Src1Var = llvm::dyn_cast<Variable>(Src1)) {
     // If src1 is a register, it should always be r32.
     if (Src1Var->hasReg()) {
-      Src1Var->asType(IceType_i32).emit(Func);
+      Src1Var->asType(IceType_i32)->emit(Func);
     } else {
       Src1Var->emit(Func);
     }

src/IceOperand.cpp

@@ -126,13 +126,16 @@ IceString Variable::getName(const Cfg *Func) const {
   return "__" + std::to_string(getIndex());
 }
 
-Variable Variable::asType(Type Ty) {
+Variable *Variable::asType(Type Ty) {
   // Note: This returns a Variable, even if the "this" object is a
   // subclass of Variable.
-  Variable V(kVariable, Ty, Number);
-  V.NameIndex = NameIndex;
-  V.RegNum = RegNum;
-  V.StackOffset = StackOffset;
+  if (!ALLOW_DUMP || getType() == Ty)
+    return this;
+  Variable *V = new (getCurrentCfgAllocator()->Allocate<Variable>())
+      Variable(kVariable, Ty, Number);
+  V->NameIndex = NameIndex;
+  V->RegNum = RegNum;
+  V->StackOffset = StackOffset;
   return V;
 }
 

src/IceOperand.h

@@ -87,7 +87,6 @@ public:
 protected:
   Operand(OperandKind Kind, Type Ty)
       : Ty(Ty), Kind(Kind), NumVars(0), Vars(NULL) {}
-  Operand(Operand &&O) = default;
 
   const Type Ty;
   const OperandKind Kind;
@@ -362,8 +361,9 @@ public:
 
 private:
   typedef std::pair<InstNumberT, InstNumberT> RangeElementType;
-  // Assume a common case of 2 or fewer segments per live range.
-  typedef llvm::SmallVector<RangeElementType, 2> RangeType;
+  // RangeType is arena-allocated from the Cfg's allocator.
+  typedef std::vector<RangeElementType, CfgLocalAllocator<RangeElementType>>
+  RangeType;
   RangeType Range;
   RegWeight Weight;
   // TrimmedBegin is an optimization for the overlaps() computation.
@@ -385,7 +385,6 @@ Ostream &operator<<(Ostream &Str, const LiveRange &L);
 class Variable : public Operand {
   Variable(const Variable &) = delete;
   Variable &operator=(const Variable &) = delete;
-  Variable(Variable &&V) = default;
 
 public:
   static Variable *create(Cfg *Func, Type Ty, SizeT Index) {
@@ -466,9 +465,9 @@ public:
   // Creates a temporary copy of the variable with a different type.
   // Used primarily for syntactic correctness of textual assembly
   // emission.  Note that only basic information is copied, in
-  // particular not DefInst, IsArgument, Weight, LoVar, HiVar,
-  // VarsReal.
-  Variable asType(Type Ty);
+  // particular not IsArgument, IsImplicitArgument, IgnoreLiveness,
+  // RegNumTmp, Weight, Live, LoVar, HiVar, VarsReal.
+  Variable *asType(Type Ty);
 
   void emit(const Cfg *Func) const override;
   using Operand::dump;
@@ -479,9 +478,6 @@ public:
     return Kind >= kVariable && Kind <= kVariable_Num;
   }
 
-  // The destructor is public because of the asType() method.
-  ~Variable() override {}
-
 protected:
   Variable(OperandKind K, Type Ty, SizeT Index)
       : Operand(K, Ty), Number(Index), NameIndex(Cfg::IdentifierIndexInvalid),
@@ -492,6 +488,7 @@ protected:
     Vars[0] = this;
     NumVars = 1;
   }
+  ~Variable() override {}
   // Number is unique across all variables, and is used as a
   // (bit)vector index for liveness analysis.
   const SizeT Number;

src/IceTargetLoweringX8632.cpp

@@ -1026,8 +1026,8 @@ template <typename T> void TargetX8632::emitConstantPool() const {
 
 void TargetX8632::emitConstants() const {
   // Note: Still used by emit IAS.
-  emitConstantPool<PoolTypeConverter<float> >();
-  emitConstantPool<PoolTypeConverter<double> >();
+  emitConstantPool<PoolTypeConverter<float>>();
+  emitConstantPool<PoolTypeConverter<double>>();
 
   // No need to emit constants from the int pool since (for x86) they
   // are embedded as immediates in the instructions.

src/PNaClTranslator.cpp

@@ -1066,7 +1066,7 @@ public:
         Timer(Ice::TimerStack::TT_parseFunctions, getTranslator().getContext()),
         Func(isIRGenerationDisabled()
                  ? nullptr
-                 : new Ice::Cfg(getTranslator().getContext())),
+                 : Ice::Cfg::create(getTranslator().getContext())),
         CurrentBbIndex(0), FcnId(Context->getNextFunctionBlockValueID()),
         FuncDecl(Context->getFunctionByID(FcnId)),
         CachedNumGlobalValueIDs(Context->getNumGlobalIDs()),

src/assembler.h

@@ -24,9 +24,7 @@
 #define SUBZERO_SRC_ASSEMBLER_H
 
 #include "IceDefs.h"
-
 #include "IceFixups.h"
-#include "llvm/Support/Allocator.h"
 
 namespace Ice {
 
@@ -232,7 +230,7 @@ public:
   void emitIASBytes(GlobalContext *Ctx) const;
 
 private:
-  llvm::BumpPtrAllocator Allocator;
+  ArenaAllocator Allocator;
 
 protected:
   AssemblerBuffer buffer_;