BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4360
BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4077
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1783893002 .

Patch from John Porto <jpp@chromium.org>.

This reverts commit 5526c171 (https://codereview.chromium.org/1778663003) and implements it per jpp's suggestion.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1780773003 .

Additionally, refactor "GetObjdumpCmd" and "GetObjcopyCmd".

BUG=https://bugs.chromium.org/p/nativeclient/issues/detail?id=4361
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1777103002 .

1. Subzero constructs many strings based in part on function name.  When function names are not present (as in properly finalized pexes), they are synthesized as something like "Function12345".  We can shorten these strings to e.g. "F12345" by using --default-function-prefix=F .  Similar for global variable names.

Using short strings makes it much less likely to have to use malloc.  As such, we force that to be the default in the browser translator build.  For perf-testing the command-line version, the user can just add the option manually for now.

Ultimately, we should avoid use of strings in this way.

2. The register allocator uses a few instances of llvm::SmallVector that are sized too small and therefore end up using malloc.  This can be fixed in a clean way, and there is a TODO for it, but in the meantime we just bump the size.

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4360
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1776343004 .

le32-nacl-objdump has been deprecated, and should no longer be used.
Instead, "arm-nacl-objdump" is being used. "le32-nacl-objdump" used to be
a hard link to "arm-nacl-objdump", but has since been deleted in NaCl's
"toolchain_build_pnacl.py" script.

R=phosek@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/1776843002 .

BUG= none
TBR=jpp@chromium.org

Review URL: https://codereview.chromium.org/1778663003 .

The ConstantRelocatable objects for pushing local labels are allocated from the Assembler arena, and are no longer pooled, which restricts the memory growth from sandboxing x86-64 calls.

Because the Assembler arena is destroyed while the fixups are still active, these fixups have to be fixed up by holding a pointer to the symbol rather than the constant.

On the 10MB test pexe, the overall growth by the end is ~20MB, instead of ~130MB as before.

This also partially fixes an existing bug with arm32/nonsfi/iasm, exposed by running cross tests and forcing iasm output.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1773503003 .

The previous implementation was charging about 24% more time that it
should to the function parser. The cause was that the time to "queue" the
parsed functions, and the time to emit the assembled code (again
including "queue" time) was not accounted for. About 15% was going to
queuing costs, and 7% to emitting the ELF file.

This CL adds timing of function translateFunctions, which captures most
of the queueing costs, and timing for each of the major ELF emission
functions (emitELF). This allows the corresponding costs to be better
bucketed, and not charged to the time it takes to parse functions in
bitcode files.

Bug=None

R=jpp@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/1775603002 .

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4076
R=eholk@chromium.org, kschimpf@google.com, stichnot@chromium.org

Review URL: https://codereview.chromium.org/1768823002 .

For the Om1 spec tests, we were generating commands like this:

  ./pydir/szbuild_spec2k.py -v -Om1 --target=x8632 -O2 --filetype=obj 253.perlbmk

where -Om1 and -O2 both appear.  Python's argparse allows an argument to be set multiple times, and the last one would win, i.e. -O2 is used instead of -Om1.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1766603002 .

This cleans up the whole --prefix handling mechanism, so that all mangling is done based on logic in the parser instead of scattered all over the code base.  Another nice side effect is that it allows some getName() style functions to return a const string reference instead of a string copy.

Also, moves ClFlags into a static field of GlobalContext, i.e. makes it global, so that these constant flags can be accessed without having to plumb a GlobalContext object.  Note that some of the ClFlags and ClFlagsExtra plumbing in the compile server classes could be simplified to directly use the corresponding static fields, but this is left mostly as is for now, for when we do a proper separation between core and supplemental flags.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1766233002 .

With "-verbose cpool", at the end it dumps counts of the number of times each pooled constant was looked up.  The output is grouped by type and sorted by constant value, but it's easy to run it through "sort -nr" to see it ordered by popularity.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1768173002 .

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4076
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1755333002 .

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4076
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1747243003 .

BUG= none
R=eholk@chromium.org, jpp@chromium.org

Review URL: https://codereview.chromium.org/1747333002 .

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4076
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1745393002 .

There were a lot of unnecessary copying and resizing and sloppiness in the use of BitVector for liveness analysis.  This tries to reduce the amount of copying and associated memory allocation.

Also, works around a ConstantRelocatable hashing problem that was causing a huge slowdown in MINIMAL mode for the vector_shuffle scons test.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1746613002 .

BUG= None
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1741733003 .

The problem is that when switch lowering decides to use a JumpTable and the switch variable is i64 on x86-64, the lowering tries to movzx the i64 variable into an i32 variable, and the Movzx ctor asserts.

This happens when translating pnacl-llc.pexe, but luckily it is also triggered in the existing adv-switch.ll test.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1743133002 .

One is rarely interested in seeing dump output regarding global initializers, so we add an additional verbose flag, and don't include it in the "-verbose most" set.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1744913002 .

Along with the nexe equivalents of pnacl-sz, also build the browser-integrated versions that can be dropped in as replacements.  Note that these are not exact replacements, as they likely support all targets.

In "make presubmit", instead build sandboxed translators in DEBUG=1 mode, so they can build in parallel with the assembler unit tests.

Add a "make bloat-sb" target that does the "make bloat" equivalent for the sandboxed translators.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1746593002 .

This reverts commit 352db935 from https://codereview.chromium.org/1740033002/ .

That commit broke "make -f Makefile.standalone check-lit check-xtest".

BUG= none
R=smklein@chromium.org

Review URL: https://codereview.chromium.org/1745783002 .

Adds "-verbose mem" to enable printing the amount of CfgLocalAllocator memory allocated, each time Cfg::dump() is called.

"-verbose mem,status" is a good way to get summary info.

BUG= none
R=kschimpf@google.com

Review URL: https://codereview.chromium.org/1743043002 .

This CL updates "isPNaClABIExternalName" -- Subzero checked for the symbol
"__pnacl_pso_root" as a function, but it is a declaration, and should be
checked as one. Additionally, when the PNaClTranslator is verifying the linkage
of declarations, allow "__pnacl_pso_root" to be flipped to external as a special
case.

TEST=pnacl-translate -pso --use-sz -arch x86-32-nonsfi test_pll.final.pso
BUG=https://bugs.chromium.org/p/nativeclient/issues/detail?id=4351
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1740033002 .

BUG=

Review URL: https://codereview.chromium.org/1742833002 .

1) Clones llvm::BitVector, and makes it Allocator aware (using the
CfgLocalAllocator<>)

2) Uses mallopt to set the malloc granularity. The default granularity
is too small, which forces too many mmap calls.

BUG=
R=sehr@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/1738683003 .

See failed build at https://build.chromium.org/p/tryserver.nacl/builders/nacl-toolchain-win7-pnacl-x86_64/builds/3441 .

BUG= none
TBR=jpp@chromium.org

Review URL: https://codereview.chromium.org/1732233002 .

Instead of "./pnacl-sz <args>", one can run:

  ../../../run.py ./pnacl-sz.x8632.nexe <args>
or
  ../../../run.py ./pnacl-sz.x8664.nexe <args>

Hopefully the translator performance characteristics are close to that of the browser hookup.

Adds sb builds to two of the presubmit configurations.

BUG= none
R=jpp@chromium.org

Review URL: https://codereview.chromium.org/1738633002 .

Introduces Ice::SmallBitVector.

Modifies ConstantRelocatables so that known offsets (i.e.,
not offsets to the code stream) do no require GlobalContext
allocations.

Modifies Cfg-local containers to use the CfgLocalAllocator.

BUG=
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1738443002 .

TBR=jpp@chromium.org

BUG= none

Review URL: https://codereview.chromium.org/1730263002 .

This is part of ARM patch:
https://codereview.chromium.org/1151663004/

BUG=
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1724643002 .

Patch from Reed Kotler <rkotlerimgtec@gmail.com>.

Implement 64 bit multiply in mips32 and, in addition, add the lo/hi registers which are also used for other 64 bit math such as div, rem.

BUG=
R=jpp@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/1716483003 .

Patch from Reed Kotler <rkotlerimgtec@gmail.com>.

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4076
R=eholk@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/1708753002 .

This removes most of the #ifndef ARM32 directives so we get more
thorough testing. It still uses fewer iterations due to running on
Qemu, but I did manually check to be sure that the tests pass without
any special casing at all.

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4076
R=jpp@chromium.org, kschimpf@google.com

Review URL: https://codereview.chromium.org/1708903002 .

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4076
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1685253003 .

With this change, we pass the select crosstest.

Since this would have introduced a three-argument version of
scalarizeInstruction, I decided to generalize it using templates.

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4076
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1683243003 .

X8664_STACK_HACK was an intrusive way to get the crosstests to run
while x32 support was not implemented in llvm. The hack is longer
needed.

R=eholk@chromium.org

Review URL: https://codereview.chromium.org/1706883003 .

BUG= https://bugs.chromium.org/p/nativeclient/issues/detail?id=4334
R=eholk@chromium.org, stichnot@chromium.org

Review URL: https://codereview.chromium.org/1697263007 .

It's troublesome that we have to type (this) in several places
but I don't see a away around it yet and it does not add to the line
of code count at all to have to do this; it's just not aesthetic to me.

If this idea is okay, I can make all the similar changes
in the Mips Subzero port.

BUG=

Review URL: https://codereview.chromium.org/1661233002 .

Patch from Reed Kotler <rkotlerimgtec@gmail.com>.

BUG=  https://bugs.chromium.org/p/nativeclient/issues/detail?id=4076
R=stichnot@chromium.org

Review URL: https://codereview.chromium.org/1409863006 .