Preliminary linking tests, seems to show that the linker
and objcopy are happy to use 'em on spec2k, and the
result runs! (Had to be careful to clobber the old .s
and .o files to make it's testing the right copy).

Haven't tried crosstests yet.

BUG=none
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/889613004

This also implicitly applies to szbuild_spec2k.py.

BUG= none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/892803002

Updates of current-function and cumulative stats are done entirely in TLS.  At the end, cumulative stats are merged across all threads' TLS into the global cumulative stats.

Printing of cumulative stats after every function is removed, since there's very little value from that.  It was probably done in the first place just to give partial cumulative information in the face of crashes or assertion failures.

BUG= none
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/887213002

MinGW's GCC 4.8.1 was sad because SectionType was shadowing the other SectionType. Also, the enum's values are in the ELFObjectWriter namespace, not ELFObjectWriter::SectionType.

R=stichnot@chromium.org, jvoung@chromium.org
BUG= Windows build is sad

Review URL: https://codereview.chromium.org/891953002

Now that multithreaded parsing and translation is in place, timer operations have to be made thread-local.  After the non-main threads end, their thread-local timer data needs to be merged into the global timer data, which resides in the GlobalContext object.  The merge is a bit tricky because the internal timer stack structure is built up dynamically as items are pushed and popped.  Two threads may have radically different timing data:

1. The parser thread profile is completely different from a translator thread.

2. For -timing-funcs, two translator threads hold data for entirely different sets of functions.

A bit more tweaking will need to be done to make the timing output fully usable in a multithreaded run.  Because of multiple threads, times may add up to >100%.  Also, time spent blocked is being "unfairly" attributed to the caller of the blocking operation - we should either count the user time instead of wall-clock time, or add a special timer marker for blocking locking operations.

BUG= none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/878383004

The problem showed up after the link step failed, in which case $(OBJDIR)/llvm2ice was deleted but the ./llvm2ice symlink still existed.  A subsequent "make check-lit" or "make check" would fail, so the basic "make" would have to be done first.

BUG= none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/887873002

The local symbol relocations are a bit different from
llvm-mc, which are section-relative. E.g., instead "bytes",
it will be ".data + offsetof(bytes, .data)". So the
contents of the text/data/rodata sections can also differ
since the offsets written in place are different.

Still need to fill the symbol table with undefined
symbols (e.g., memset, and szrt lib functions) before
trying to link.

BUG=none
R=kschimpf@google.com, stichnot@chromium.org

Review URL: https://codereview.chromium.org/874353006

__attribute__((aligned(MaxCacheLineSize))) triggers a GCC bug because enum {MaxCacheLineSize = 64 }; isn't constant enough. Adding zero to it makes it that much more constant.

R= stichnot@chromium.org
BUG= https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55382

Review URL: https://codereview.chromium.org/867483004

The period, it was missing

R= stichnot@chromium.org
BUG= none
.

Review URL: https://codereview.chromium.org/888473002

<mutex> is already included from IceDefs.h (where GlobalLockType is defined) but <condition_variable> isn't included anywhere. It's probably included indirectly in some standard libraries and not others, causing build failures on Windows.

TBR= stichnot@chromium.org
BUG= none

Review URL: https://codereview.chromium.org/884283002

Cleans up code by removing unnecessary fields/data structures in top-level parser of Subzero. In particular:

1) Uses FunctionDeclarationList.size() instead of NumFunctionIds.

2) Removes the need for vector DefiningFunctionDeclarationList. Instead uses an (incremented) index NextDefiningFunctionID into vector
FunctionDeclarationList.

BUG=None
R=jvoung@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/883493002

BUG= none
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/865093003

This also requires modifying the ICE_CACHELINE_BOUNDARY macro to avoid a warning about anonymous structs:

src/IceUtils.h:132:3: warning: anonymous structs are a GNU extension [-Wgnu-anonymous-struct]

BUG= none
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/883983002

GCC 4.8.1 is sad;

There are extra semicolons in Subzero;

It removes the semicolons or it gets the build warning hose again;^H

R=stichnot@chromium.org
BUG= none

Review URL: https://codereview.chromium.org/882743003

There are two problems with "make format" and "make format-diff" in
Makefile.standalone:

1. You have to make sure clang-format and clang-format-diff.py are
available in $PATH.

2. Different users may have different versions installed (even for the
same user on different machines), leading to whitespace wars.  Can't we
all just get along?

Since the normal LLVM build that Subzero depends on also exposes and
builds clang-format and friends, we might as well use it.  The
clang-format binary is found in $LLVM_BIN_PATH, and clang-format-diff.py
is found relative to $LLVM_SRC_PATH.  As long as the user's LLVM build
is fairly up to date, whitespace wars are unlikely.

Given this, there's a much higher incentive to use "make format"
regularly instead of "make format-diff".  In particular, inline comments
on variable/field declaration lists can get lined up more nicely by
looking at the entire context, rather than the small diff window.

BUG= none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/877003003

Provides a single-producer, multiple-consumer translation queue where the number of translation threads is given by the -threads=N argument.  The producer (i.e., bitcode parser) blocks if the queue size is >=N, in order to control the memory footprint.  If N=0 (which is the default), execution is purely single-threaded.  If N=1, there is a single translation thread running in parallel with the parser thread.  "make check" succeeds with the default changed to N=1.

Currently emission is also done by the translation thread, which limits scalability since the emit stream has to be locked.  Also, since the ELF writer stream is not locked, it won't be safe to use N>1 with the ELF writer.  Furthermore, for N>1, emitted function ordering is nondeterministic and needs to be recombobulated.  This will all be fixed in a follow-on CL.

The -timing option is broken for N>0.  This will be fixed in a follow-on CL.

Verbose flags are now managed in the Cfg instead of (or in addition to) the GlobalContext, due to the -verbose-focus option which wants to temporarily change the verbose level for a particular function.

TargetLowering::emitConstants() and related methods are changed to be static, so that a valid TargetLowering object isn't required.  This is because the TargetLowering object wants to hold a valid Cfg, and none really exists after all functions are translated and the constant pool is ready for emission.

The Makefile.standalone now has a TSAN=1 option to enable ThreadSanitizer.

BUG= none
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/870653002

Manages thread_local pointer fields through a set of macros.  If ICE_THREAD_LOCAL_HACK is defined, the thread_local definitions and accesses are defined in terms of pthread operations.  This assumes that the underlying std::thread library is based on pthread.

BUG= none
R=jfb@chromium.org, jvoung@chromium.org

Review URL: https://codereview.chromium.org/872933002

This reduces the number of conditionals, and will more closely reflect
the structure of the ELF writer's version of the same thing.
Without fdata-sections, the ELF writer version will have to batch all
initializers of a certain type so that they can be contiguous on the file
and the overall alignment can be determined.

A downside of this is that, .s files will be different from llc's output.
The spec .o and executables are identical before/after the change.

BUG=none
R=kschimpf@google.com, stichnot@chromium.org

Review URL: https://codereview.chromium.org/870123003

MacOS doesn't support the thread_local keyword until 10.7 or later, and our bots run 10.6.

Who knows whether Visual Studio supports it yet.

In the meantime, use the old-style syntax.

BUG= https://codereview.chromium.org/873443004/
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/865973006

The following CL enables -Werror:
  https://codereview.chromium.org/863093002/

There were two warnings left in our subzero build:
 - Dead default cases because all of an enum's values were handled by the switch.
 - Use of C99 VLA.

R=stichnot@chromium.org
TEST= make check
BUG= none

Review URL: https://codereview.chromium.org/862853003

The previous code did not do this. Also localizes the lock to when
the error is actually printed.

Note: requires https://codereview.chromium.org/865963002

BUG=None
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/864383002

Elements were added to this vector, but never inspected, so it is
essentially a useless field.  Plus, the removal allows us to remove a
couple of friend declarations.

BUG=none
R=kschimpf@google.com

Review URL: https://codereview.chromium.org/814163004

This just gets the locking in place.  Actual multithreading will be added later.

Mutexes are added for accessing the GlobalContext allocator, the constant pool, the stats data, and the profiling timers.  These are managed via the LockedPtr<> helper.  Finer grain locks on the constant pool may be added later, i.e. a separate lock for each data type.

An vector of pointers to TLS objects is added to GlobalContext.  Each new thread will get its own TLS object, whose address is added to the vector.  (After threads complete, things like stats can be combined by iterating over the vector.)

The dump/emit streams are guarded by a separate lock, to avoid fine-grain interleaving of output by multiple threads.  E.g., lock the streams, emit an entire function, and unlock the streams.  This works for dumping too, though dump output for different passes on the same function may be interleaved with that of another thread.  There is an OstreamLocker helper class to keep this simple.

CodeStats is made an inner class of GlobalContext (this was missed on a previous CL).

BUG= none
R=jfb@chromium.org, jvoung@chromium.org, kschimpf@google.com

Review URL: https://codereview.chromium.org/848193003

BUG=None
R=jvoung@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/848473002

This hasn't been used in a very long time, and there's no intention of using it again.

Originally there was the idea of a "fast" block-local register allocator for an O1-like configuration, which would allocate registers for infinite-weight temporaries during target lowering, using a "local register manager".  This verbose option was for tracing execution of this register manager.  However, by now it seems unlikely that this would do a better/faster job than the current Om1 register allocation approach, which reuses the linear-scan code quite effectively and does very well at separation of concerns.  So adios IceV_RegManager!

BUG= none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/831663008

Pass the full assembler pointer to the elf writer, so
that it has access to both the text buffer and the fixups.

Remove some child classes of AssemblerFixups. They didn't
really do much, and were pretty much identical to the
original AssemblerFixup class. Dart had a virtual method
for fixups to do necessary patching, but we currently
don't do the patching and just emit the relocations.
TODO see if patching is more efficient than writing out
relocations and letting the linker do the work.

This CL also makes AssemblerFixups POD.

Change the fixup kind to be a plain unsigned int, which
the target can fill w/ target/container-specific values.

Move the fwd declaration of Assembler to IceDefs and remove
the others. Do similar for fwd declaration refactoring for
ELFWriter.

Make the createAssembler method return a std::unique_ptr.

BUG=none
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/828873002

BUG= none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/848603002

BUG= none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/846763002

This avoids doing getConstantSym to avoid hitting the global
context's getConstantSym during emitIAS(), which may be desirable for
multi-threading, since each function's emitIAS() should be able to happen
on a separate thread.

The stringification is moved till later, so it still happens, just without
creating a constant relocatable w/ offset of 0.

This ends up tickling an issue where -O0 on 252.eon now gets 2x as many
page faults, and I'm not sure exactly why. This makes the overall time
higher, though emit time is lower.

When translating with -O2 # of page faults is about the same before/after,
so that oddness is restricted to O0.

Before this change, tweaking the slab size at O0 doesn't
seem to affect as drastically as 2x swings either.

To work around this, I turned the slab size of the assembler down to 32KB.

===

Move all the .L$type$poolid into a function (replacing getPoolEntryID).

BUG=none
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/837553009

Extends the NaCl bitcode munger so that the PNaClTranslator parser
can be applied to the defined sequence of record values.

BUG=None
R=jvoung@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/800883006

Create the section, write the data and define the symbols in
the symbol table.

BUG=none
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/837393005

1. Use a reverse_range() adaptor for range-based reverse iteration through containers.

2. Remove the emitting of the commented llvm-mc command line.

3. Remove a few TODOs.

4. For commented-out declarations within a class T like this:
   // T(const T&) = delete;
   // T &operator=(const T &) = delete;
Replace them with this:
   T(const T&) = default;
   T &operator=(const T &) = default;
And try to keep them private where possible.

5. Make LivenessNode and TimerTreeNode into internal classes.

BUG= none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/830303003

It doesn't seem to be used?

BUG=none
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/840343002

BUG= none
R=jvoung@chromium.org

Review URL: https://codereview.chromium.org/812023003

This reestablishes C++11 features lost after switching to llvm::ilist<> in two previous CLs:
https://codereview.chromium.org/709533002/
https://codereview.chromium.org/794923002/

BUG= none
R=jfb@chromium.org, jvoung@chromium.org

Review URL: https://codereview.chromium.org/819403002

This is limited to the src directory, and not (yet) the crosstest directory.

BUG= none
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/814353002

In particular, node lists for in and out edges of a CfgNode, and the live range segment list in a Variable.  This is done by making the Cfg allocator globally available through TLS, and providing the STL containers with an allocator struct that uses this.

This also cleans up some other allocation-related issues:

* The allocator is now hung off the Cfg via a pointer, rather than being embedded into the Cfg.  This allows a const Cfg pointer to be stored in TLS while still allowing its allocator to be mutated.

* Cfg is now created via a static create() method.

* The redundant Cfg::allocateInst<> methods are removed.

* The Variable::asType() method allocates a whole new Variable from the Cfg arena, rather than allocating it on the stack, removing the need for the move constructor in Variable and Operand.  This is OK since asType() is only used for textual asm emission.

* The same 1MB arena allocator is now used by the assembler as well.  The fact that it wasn't changed over to be the same as Cfg and GlobalContext was an oversight.  (It turns out this adds ~3MB to the translator memory footprint, so that could be tuned later.)

BUG= none
R=jfb@chromium.org, jvoung@chromium.org

Review URL: https://codereview.chromium.org/802183004

Randomize the order that registers appear in the free list.  Only
randomize fully "equivalent" registers to ensure no extra spills.

This adds the -randomize-regalloc option.

This is a continuation of https://codereview.chromium.org/456033003/ which Matt owns.

BUG= none
R=jfb@chromium.org

Review URL: https://codereview.chromium.org/807293003

Removes the need to model LLVM types from the minimal subzero build.
It isn't removed from the nonminimal build because IceConverter still needs
to be able to convert LLVM types to corresponding Ice types.

Note that this CL reduces the size of Release+Min/llvm2ice (after
strip) to about 638K bytes.

BUG=None
R=jvoung@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/805943002

Removes cast instruction checks (in PNaClTranslator.cpp) that used
LLVM utilities to use locally defined methods instead. Remove the need
to call naclbitc::DecodeCastOpcode and CastInst::castIsValid.

Also removes two more calls to convertToLLVMType.

BUG= None
R=jvoung@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/794823002