Skip to content

G
glslang
Commit 3c4a2762 by John Kessenich
Options

SPIR-V compression: Requires rerunning CMake. Adds a standalone tool for…

SPIR-V compression: Requires rerunning CMake. Adds a standalone tool for running the SPV compression. git-svn-id: https://cvs.khronos.org/svn/repos/ogl/trunk/ecosystem/public/sdk/tools/glslang@31232 e7fa87d3-cd2b-0410-9028-fcbf551c1848
parent 40e39118
Showing with 1654 additions and 1174 deletions
README-spirv-remap.txt 0 → 100644
VERSION
--------------------------------------------------------------------------------
spirv-remap 0.97
INTRO:
--------------------------------------------------------------------------------
spirv-remap is a utility to improve compression of SPIRV binary files via
entropy reduction, plus optional stripping of debug information and
load/store optimization. It transforms SPIRV to SPIRV, remapping IDs. The
resulting modules have an increased ID range (IDs are not as tightly packed
around zero), but will compress better when multiple modules are compressed
together, since compressor's dictionary can find better cross module
commonality.
Remapping is accomplished via canonicalization. Thus, modules can be
compressed one at a time with no loss of quality relative to operating on
many modules at once. The command line tool operates on multiple modules
only in the trivial repetition sense, for ease of use. The remapper API
only accepts a single module at a time.
There are two modes of use: command line, and a C++11 API. Both are
described below.
spirv-remap is currently in an alpha state. Although there are no known
remapping defects, it has only been exercised on one real world game shader
workload.
FEEDBACK
--------------------------------------------------------------------------------
Report defects, enhancements requests, code improvements, etc to:
spvremapper@lunarg.com
COMMAND LINE USAGE:
--------------------------------------------------------------------------------
Examples are given with a verbosity of one (-v), but more verbosity can be
had via -vv, -vvv, etc, or an integer parameter to --verbose, such as
"--verbose 4". With no verbosity, the command is silent and returns 0 on
success, and a positive integer error on failure.
Pre-built binaries for several OSs are available. Examples presented are
for Linux. Command line arguments can be provided in any order.
1. Basic ID remapping
Perform ID remapping on all shaders in "*.spv", writing new files with
the same basenames to /tmp/out_dir.
spirv-remap --map all --input *.spv --output /tmp/out_dir
2. Perform all possible size reductions
spirv-remap-linux-64 --do-everything --input *.spv --output /tmp/out_dir
Note that --do-everything is a synonym for:
--map all --dce all --opt all --strip all
API USAGE:
--------------------------------------------------------------------------------
The public interface to the remapper is defined in SPIRV/SPVRemapper.h as follows:
namespace spv {
class spirvbin_t
{
public:
enum Options { ... };
spirvbin_t(int verbose = 0); // construct
// remap an existing binary in memory
void remap(std::vector<std::uint32_t>& spv, std::uint32_t opts = Options::DO_EVERYTHING);
// Type for error/log handler functions
typedef std::function<void(const std::string&)> errorfn_t;
typedef std::function<void(const std::string&)> logfn_t;
// Register error/log handling functions (can be c/c++ fn, lambda fn, or functor)
static void registerErrorHandler(errorfn_t handler) { errorHandler = handler; }
static void registerLogHandler(logfn_t handler) { logHandler = handler; }
};
} // namespace spv
The class definition is in SPVRemapper.cpp.
remap() accepts an std::vector of SPIRV words, modifies them per the
request given in 'opts', and leaves the 'spv' container with the result.
It is safe to instantiate one spirvbin_t per thread and process a different
SPIRV in each.
The "opts" parameter to remap() accepts a bit mask of desired remapping
options. See REMAPPING AND OPTIMIZATION OPTIONS.
On error, the function supplied to registerErrorHandler() will be invoked.
This can be a standard C/C++ function, a lambda function, or a functor.
The default handler simply calls exit(5); The error handler is a static
members, so need only be set up once, not once per spirvbin_t instance.
Log messages are supplied to registerLogHandler(). By default, log
messages are eaten silently. The log handler is also a static member.
BUILD DEPENDENCIES:
--------------------------------------------------------------------------------
1. C++11 compatible compiler
2. cmake
3. glslang
BUILDING
--------------------------------------------------------------------------------
The standalone remapper is built along side glslangValidator through its
normal build process.
REMAPPING AND OPTIMIZATION OPTIONS
--------------------------------------------------------------------------------
API:
These are bits defined under spv::spirvbin_t::Options::, and can be
bitwise or-ed together as desired.
MAP_TYPES = canonicalize type IDs
MAP_NAMES = canonicalize named data
MAP_FUNCS = canonicalize function bodies
DCE_FUNCS = remove dead functions
DCE_VARS = remove dead variables
DCE_TYPES = remove dead types
OPT_LOADSTORE = optimize unneeded load/stores
MAP_ALL = (MAP_TYPES | MAP_NAMES | MAP_FUNCS)
DCE_ALL = (DCE_FUNCS | DCE_VARS | DCE_TYPES)
OPT_ALL = (OPT_LOADSTORE)
ALL_BUT_STRIP = (MAP_ALL | DCE_ALL | OPT_ALL)
DO_EVERYTHING = (STRIP | ALL_BUT_STRIP)
SPIRV/SPVRemapper.cpp
...@@ -32,11 +32,9 @@ ...@@ -32,11 +32,9 @@
//ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE //ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
//POSSIBILITY OF SUCH DAMAGE. //POSSIBILITY OF SUCH DAMAGE.
// //
#include "SPVRemapper.h"
#include "doc.h"
/* -*-mode:c++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 3 -*- */ #include "SPVRemapper.h"
#include "doc.h"
#if !defined (use_cpp11) #if !defined (use_cpp11)
// ... not supported before C++11 // ... not supported before C++11
...@@ -45,1178 +43,1181 @@ ...@@ -45,1178 +43,1181 @@
#include <algorithm> #include <algorithm>
#include <cassert> #include <cassert>
namespace spv { namespace spv {
// By default, just abort on error. Can be overridden via RegisterErrorHandler // By default, just abort on error. Can be overridden via RegisterErrorHandler
spirvbin_t::errorfn_t spirvbin_t::errorHandler = [](const std::string&) { exit(5); }; spirvbin_t::errorfn_t spirvbin_t::errorHandler = [](const std::string&) { exit(5); };
// By default, eat log messages. Can be overridden via RegisterLogHandler // By default, eat log messages. Can be overridden via RegisterLogHandler
spirvbin_t::logfn_t spirvbin_t::logHandler = [](const std::string&) { }; spirvbin_t::logfn_t spirvbin_t::logHandler = [](const std::string&) { };
// This can be overridden to provide other message behavior if needed // This can be overridden to provide other message behavior if needed
void spirvbin_t::msg(int minVerbosity, int indent, const std::string& txt) const void spirvbin_t::msg(int minVerbosity, int indent, const std::string& txt) const
{ {
if (verbose >= minVerbosity) if (verbose >= minVerbosity)
logHandler(std::string(indent, ' ') + txt); logHandler(std::string(indent, ' ') + txt);
} }
// hash opcode, with special handling for OpExtInst // hash opcode, with special handling for OpExtInst
std::uint32_t spirvbin_t::asOpCodeHash(int word) std::uint32_t spirvbin_t::asOpCodeHash(int word)
{ {
const spv::Op opCode = asOpCode(word); const spv::Op opCode = asOpCode(word);
std::uint32_t offset = 0; std::uint32_t offset = 0;
switch (opCode) { switch (opCode) {
case spv::OpExtInst: case spv::OpExtInst:
offset += asId(word + 4); break; offset += asId(word + 4); break;
default: default:
break; break;
} }
return opCode * 19 + offset; // 19 = small prime return opCode * 19 + offset; // 19 = small prime
} }
spirvbin_t::range_t spirvbin_t::literalRange(spv::Op opCode) const spirvbin_t::range_t spirvbin_t::literalRange(spv::Op opCode) const
{ {
static const int maxCount = 1<<30; static const int maxCount = 1<<30;
switch (opCode) { switch (opCode) {
case spv::OpTypeFloat: // fall through... case spv::OpTypeFloat: // fall through...
case spv::OpTypePointer: return range_t(2, 3); case spv::OpTypePointer: return range_t(2, 3);
case spv::OpTypeInt: return range_t(2, 4); case spv::OpTypeInt: return range_t(2, 4);
case spv::OpTypeSampler: return range_t(3, 8); case spv::OpTypeSampler: return range_t(3, 8);
case spv::OpTypeVector: // fall through case spv::OpTypeVector: // fall through
case spv::OpTypeMatrix: // ... case spv::OpTypeMatrix: // ...
case spv::OpTypePipe: return range_t(3, 4); case spv::OpTypePipe: return range_t(3, 4);
case spv::OpConstant: return range_t(3, maxCount); case spv::OpConstant: return range_t(3, maxCount);
default: return range_t(0, 0); default: return range_t(0, 0);
} }
} }
spirvbin_t::range_t spirvbin_t::typeRange(spv::Op opCode) const spirvbin_t::range_t spirvbin_t::typeRange(spv::Op opCode) const
{ {
static const int maxCount = 1<<30; static const int maxCount = 1<<30;
if (isConstOp(opCode)) if (isConstOp(opCode))
return range_t(1, 2); return range_t(1, 2);
switch (opCode) { switch (opCode) {
case spv::OpTypeVector: // fall through case spv::OpTypeVector: // fall through
case spv::OpTypeMatrix: // ... case spv::OpTypeMatrix: // ...
case spv::OpTypeSampler: // ... case spv::OpTypeSampler: // ...
case spv::OpTypeArray: // ... case spv::OpTypeArray: // ...
case spv::OpTypeRuntimeArray: // ... case spv::OpTypeRuntimeArray: // ...
case spv::OpTypePipe: return range_t(2, 3); case spv::OpTypePipe: return range_t(2, 3);
case spv::OpTypeStruct: // fall through case spv::OpTypeStruct: // fall through
case spv::OpTypeFunction: return range_t(2, maxCount); case spv::OpTypeFunction: return range_t(2, maxCount);
case spv::OpTypePointer: return range_t(3, 4); case spv::OpTypePointer: return range_t(3, 4);
default: return range_t(0, 0); default: return range_t(0, 0);
} }
} }
spirvbin_t::range_t spirvbin_t::constRange(spv::Op opCode) const spirvbin_t::range_t spirvbin_t::constRange(spv::Op opCode) const
{ {
static const int maxCount = 1<<30; static const int maxCount = 1<<30;
switch (opCode) { switch (opCode) {
case spv::OpTypeArray: // fall through... case spv::OpTypeArray: // fall through...
case spv::OpTypeRuntimeArray: return range_t(3, 4); case spv::OpTypeRuntimeArray: return range_t(3, 4);
case spv::OpConstantComposite: return range_t(3, maxCount); case spv::OpConstantComposite: return range_t(3, maxCount);
default: return range_t(0, 0); default: return range_t(0, 0);
} }
} }
// Is this an opcode we should remove when using --strip? // Is this an opcode we should remove when using --strip?
bool spirvbin_t::isStripOp(spv::Op opCode) const bool spirvbin_t::isStripOp(spv::Op opCode) const
{ {
switch (opCode) { switch (opCode) {
case spv::OpSource: case spv::OpSource:
case spv::OpSourceExtension: case spv::OpSourceExtension:
case spv::OpName: case spv::OpName:
case spv::OpMemberName: case spv::OpMemberName:
case spv::OpLine: return true; case spv::OpLine: return true;
default: return false; default: return false;
} }
} }
bool spirvbin_t::isFlowCtrlOpen(spv::Op opCode) const bool spirvbin_t::isFlowCtrlOpen(spv::Op opCode) const
{ {
switch (opCode) { switch (opCode) {
case spv::OpBranchConditional: case spv::OpBranchConditional:
case spv::OpSwitch: return true; case spv::OpSwitch: return true;
default: return false; default: return false;
} }
} }
bool spirvbin_t::isFlowCtrlClose(spv::Op opCode) const bool spirvbin_t::isFlowCtrlClose(spv::Op opCode) const
{ {
switch (opCode) { switch (opCode) {
case spv::OpLoopMerge: case spv::OpLoopMerge:
case spv::OpSelectionMerge: return true; case spv::OpSelectionMerge: return true;
default: return false; default: return false;
} }
} }
bool spirvbin_t::isTypeOp(spv::Op opCode) const bool spirvbin_t::isTypeOp(spv::Op opCode) const
{ {
switch (opCode) { switch (opCode) {
case spv::OpTypeVoid: case spv::OpTypeVoid:
case spv::OpTypeBool: case spv::OpTypeBool:
case spv::OpTypeInt: case spv::OpTypeInt:
case spv::OpTypeFloat: case spv::OpTypeFloat:
case spv::OpTypeVector: case spv::OpTypeVector:
case spv::OpTypeMatrix: case spv::OpTypeMatrix:
case spv::OpTypeSampler: case spv::OpTypeSampler:
case spv::OpTypeFilter: case spv::OpTypeFilter:
case spv::OpTypeArray: case spv::OpTypeArray:
case spv::OpTypeRuntimeArray: case spv::OpTypeRuntimeArray:
case spv::OpTypeStruct: case spv::OpTypeStruct:
case spv::OpTypeOpaque: case spv::OpTypeOpaque:
case spv::OpTypePointer: case spv::OpTypePointer:
case spv::OpTypeFunction: case spv::OpTypeFunction:
case spv::OpTypeEvent: case spv::OpTypeEvent:
case spv::OpTypeDeviceEvent: case spv::OpTypeDeviceEvent:
case spv::OpTypeReserveId: case spv::OpTypeReserveId:
case spv::OpTypeQueue: case spv::OpTypeQueue:
case spv::OpTypePipe: return true; case spv::OpTypePipe: return true;
default: return false; default: return false;
} }
} }
bool spirvbin_t::isConstOp(spv::Op opCode) const bool spirvbin_t::isConstOp(spv::Op opCode) const
{ {
switch (opCode) { switch (opCode) {
case spv::OpConstantNullObject: error("unimplemented constant type"); case spv::OpConstantNullObject: error("unimplemented constant type");
case spv::OpConstantSampler: error("unimplemented constant type"); case spv::OpConstantSampler: error("unimplemented constant type");
case spv::OpConstantTrue: case spv::OpConstantTrue:
case spv::OpConstantFalse: case spv::OpConstantFalse:
case spv::OpConstantNullPointer: case spv::OpConstantNullPointer:
case spv::OpConstantComposite: case spv::OpConstantComposite:
case spv::OpConstant: return true; case spv::OpConstant: return true;
default: return false; default: return false;
} }
} }
const auto inst_fn_nop = [](spv::Op, int) { return false; }; const auto inst_fn_nop = [](spv::Op, int) { return false; };
const auto op_fn_nop = [](spv::Id&) { }; const auto op_fn_nop = [](spv::Id&) { };
// g++ doesn't like these defined in the class proper in an anonymous namespace. // g++ doesn't like these defined in the class proper in an anonymous namespace.
// Dunno why. Also MSVC doesn't like the constexpr keyword. Also dunno why. // Dunno why. Also MSVC doesn't like the constexpr keyword. Also dunno why.
// Defining them externally seems to please both compilers, so, here they are. // Defining them externally seems to please both compilers, so, here they are.
const spv::Id spirvbin_t::unmapped = spv::Id(-10000); const spv::Id spirvbin_t::unmapped = spv::Id(-10000);
const spv::Id spirvbin_t::unused = spv::Id(-10001); const spv::Id spirvbin_t::unused = spv::Id(-10001);
const int spirvbin_t::header_size = 5; const int spirvbin_t::header_size = 5;
spv::Id spirvbin_t::nextUnusedId(spv::Id id) spv::Id spirvbin_t::nextUnusedId(spv::Id id)
{ {
while (isNewIdMapped(id)) // search for an unused ID while (isNewIdMapped(id)) // search for an unused ID
++id; ++id;
return id; return id;
} }
spv::Id spirvbin_t::localId(spv::Id id, spv::Id newId) spv::Id spirvbin_t::localId(spv::Id id, spv::Id newId)
{ {
assert(id != spv::NoResult && newId != spv::NoResult); assert(id != spv::NoResult && newId != spv::NoResult);
if (id >= idMapL.size()) if (id >= idMapL.size())
idMapL.resize(id+1, unused); idMapL.resize(id+1, unused);
if (newId != unmapped && newId != unused) { if (newId != unmapped && newId != unused) {
if (isOldIdUnused(id)) if (isOldIdUnused(id))
error(std::string("ID unused in module: ") + std::to_string(id)); error(std::string("ID unused in module: ") + std::to_string(id));
if (!isOldIdUnmapped(id)) if (!isOldIdUnmapped(id))
error(std::string("ID already mapped: ") + std::to_string(id) + " -> " error(std::string("ID already mapped: ") + std::to_string(id) + " -> "
+ std::to_string(localId(id))); + std::to_string(localId(id)));
if (isNewIdMapped(newId)) if (isNewIdMapped(newId))
error(std::string("ID already used in module: ") + std::to_string(newId)); error(std::string("ID already used in module: ") + std::to_string(newId));
msg(4, 4, std::string("map: ") + std::to_string(id) + " -> " + std::to_string(newId)); msg(4, 4, std::string("map: ") + std::to_string(id) + " -> " + std::to_string(newId));
setMapped(newId); setMapped(newId);
largestNewId = std::max(largestNewId, newId); largestNewId = std::max(largestNewId, newId);
} }
return idMapL[id] = newId; return idMapL[id] = newId;
} }
// Parse a literal string from the SPIR binary and return it as an std::string // Parse a literal string from the SPIR binary and return it as an std::string
// Due to C++11 RValue references, this doesn't copy the result string. // Due to C++11 RValue references, this doesn't copy the result string.
std::string spirvbin_t::literalString(int word) const std::string spirvbin_t::literalString(int word) const
{ {
std::string literal; std::string literal;
literal.reserve(16); literal.reserve(16);
const char* bytes = reinterpret_cast<const char*>(spv.data() + word); const char* bytes = reinterpret_cast<const char*>(spv.data() + word);
while (bytes && *bytes) while (bytes && *bytes)
literal += *bytes++; literal += *bytes++;
return literal; return literal;
} }
void spirvbin_t::applyMap() void spirvbin_t::applyMap()
{ {
msg(3, 2, std::string("Applying map: ")); msg(3, 2, std::string("Applying map: "));
// Map local IDs through the ID map // Map local IDs through the ID map
process(inst_fn_nop, // ignore instructions process(inst_fn_nop, // ignore instructions
[this](spv::Id& id) { [this](spv::Id& id) {
id = localId(id); id = localId(id);
assert(id != unused && id != unmapped); assert(id != unused && id != unmapped);
} }
); );
} }
// Find free IDs for anything we haven't mapped // Find free IDs for anything we haven't mapped
void spirvbin_t::mapRemainder() void spirvbin_t::mapRemainder()
{ {
msg(3, 2, std::string("Remapping remainder: ")); msg(3, 2, std::string("Remapping remainder: "));
spv::Id unusedId = 1; // can't use 0: that's NoResult spv::Id unusedId = 1; // can't use 0: that's NoResult
spirword_t maxBound = 0; spirword_t maxBound = 0;
for (spv::Id id = 0; id < idMapL.size(); ++id) { for (spv::Id id = 0; id < idMapL.size(); ++id) {
if (isOldIdUnused(id)) if (isOldIdUnused(id))
continue; continue;
// Find a new mapping for any used but unmapped IDs // Find a new mapping for any used but unmapped IDs
if (isOldIdUnmapped(id)) if (isOldIdUnmapped(id))
localId(id, unusedId = nextUnusedId(unusedId)); localId(id, unusedId = nextUnusedId(unusedId));
if (isOldIdUnmapped(id)) if (isOldIdUnmapped(id))
error(std::string("old ID not mapped: ") + std::to_string(id)); error(std::string("old ID not mapped: ") + std::to_string(id));
// Track max bound // Track max bound
maxBound = std::max(maxBound, localId(id) + 1); maxBound = std::max(maxBound, localId(id) + 1);
} }
bound(maxBound); // reset header ID bound to as big as it now needs to be bound(maxBound); // reset header ID bound to as big as it now needs to be
} }
void spirvbin_t::stripDebug() void spirvbin_t::stripDebug()
{ {
if ((options & Options::STRIP) == 0) if ((options & Options::STRIP) == 0)
return; return;
// build local Id and name maps // build local Id and name maps
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
// remember opcodes we want to strip later // remember opcodes we want to strip later
if (isStripOp(opCode)) if (isStripOp(opCode))
stripInst(start); stripInst(start);
return true; return true;
}, },
op_fn_nop); op_fn_nop);
} }
void spirvbin_t::buildLocalMaps() void spirvbin_t::buildLocalMaps()
{ {
msg(2, 2, std::string("build local maps: ")); msg(2, 2, std::string("build local maps: "));
mapped.clear(); mapped.clear();
idMapL.clear(); idMapL.clear();
nameMap.clear(); nameMap.clear();
fnPos.clear(); fnPos.clear();
fnPosDCE.clear(); fnPosDCE.clear();
fnCalls.clear(); fnCalls.clear();
typeConstPos.clear(); typeConstPos.clear();
typeConstPosR.clear(); typeConstPosR.clear();
entryPoint = spv::NoResult; entryPoint = spv::NoResult;
largestNewId = 0; largestNewId = 0;
idMapL.resize(bound(), unused); idMapL.resize(bound(), unused);
int fnStart = 0; int fnStart = 0;
spv::Id fnRes = spv::NoResult; spv::Id fnRes = spv::NoResult;
// build local Id and name maps // build local Id and name maps
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
// remember opcodes we want to strip later // remember opcodes we want to strip later
if ((options & Options::STRIP) && isStripOp(opCode)) if ((options & Options::STRIP) && isStripOp(opCode))
stripInst(start); stripInst(start);
if (opCode == spv::Op::OpName) { if (opCode == spv::Op::OpName) {
const spv::Id target = asId(start+1); const spv::Id target = asId(start+1);
const std::string name = literalString(start+2); const std::string name = literalString(start+2);
nameMap[name] = target; nameMap[name] = target;
return true; return true;
} else if (opCode == spv::Op::OpFunctionCall) { } else if (opCode == spv::Op::OpFunctionCall) {
++fnCalls[asId(start + 3)]; ++fnCalls[asId(start + 3)];
} else if (opCode == spv::Op::OpEntryPoint) { } else if (opCode == spv::Op::OpEntryPoint) {
entryPoint = asId(start + 2); entryPoint = asId(start + 2);
} else if (opCode == spv::Op::OpFunction) { } else if (opCode == spv::Op::OpFunction) {
if (fnStart != 0) if (fnStart != 0)
error("nested function found"); error("nested function found");
fnStart = start; fnStart = start;
fnRes = asId(start + 2); fnRes = asId(start + 2);
} else if (opCode == spv::Op::OpFunctionEnd) { } else if (opCode == spv::Op::OpFunctionEnd) {
assert(fnRes != spv::NoResult); assert(fnRes != spv::NoResult);
if (fnStart == 0) if (fnStart == 0)
error("function end without function start"); error("function end without function start");
fnPos[fnRes] = {fnStart, start + asWordCount(start)}; fnPos[fnRes] = {fnStart, start + asWordCount(start)};
fnStart = 0; fnStart = 0;
} else if (isConstOp(opCode)) { } else if (isConstOp(opCode)) {
assert(asId(start + 2) != spv::NoResult); assert(asId(start + 2) != spv::NoResult);
typeConstPos.insert(start); typeConstPos.insert(start);
typeConstPosR[asId(start + 2)] = start; typeConstPosR[asId(start + 2)] = start;
} else if (isTypeOp(opCode)) { } else if (isTypeOp(opCode)) {
assert(asId(start + 1) != spv::NoResult); assert(asId(start + 1) != spv::NoResult);
typeConstPos.insert(start); typeConstPos.insert(start);
typeConstPosR[asId(start + 1)] = start; typeConstPosR[asId(start + 1)] = start;
} }
return false; return false;
}, },
[this](spv::Id& id) { localId(id, unmapped); } [this](spv::Id& id) { localId(id, unmapped); }
); );
} }
// Validate the SPIR header // Validate the SPIR header
void spirvbin_t::validate() const void spirvbin_t::validate() const
{ {
msg(2, 2, std::string("validating: ")); msg(2, 2, std::string("validating: "));
if (spv.size() < header_size) if (spv.size() < header_size)
error("file too short: "); error("file too short: ");
if (magic() != spv::MagicNumber) if (magic() != spv::MagicNumber)
error("bad magic number"); error("bad magic number");
// field 1 = version // field 1 = version
// field 2 = generator magic // field 2 = generator magic
// field 3 = result <id> bound // field 3 = result <id> bound
if (schemaNum() != 0) if (schemaNum() != 0)
error("bad schema, must be 0"); error("bad schema, must be 0");
} }
int spirvbin_t::processInstruction(int word, instfn_t instFn, idfn_t idFn) int spirvbin_t::processInstruction(int word, instfn_t instFn, idfn_t idFn)
{ {
const auto instructionStart = word; const auto instructionStart = word;
const unsigned wordCount = asWordCount(instructionStart); const unsigned wordCount = asWordCount(instructionStart);
const spv::Op opCode = asOpCode(instructionStart); const spv::Op opCode = asOpCode(instructionStart);
const int nextInst = word++ + wordCount; const int nextInst = word++ + wordCount;
if (nextInst > int(spv.size())) if (nextInst > int(spv.size()))
error("spir instruction terminated too early"); error("spir instruction terminated too early");
// Base for computing number of operands; will be updated as more is learned // Base for computing number of operands; will be updated as more is learned
unsigned numOperands = wordCount - 1; unsigned numOperands = wordCount - 1;
if (instFn(opCode, instructionStart)) if (instFn(opCode, instructionStart))
return nextInst; return nextInst;
// Read type and result ID from instruction desc table // Read type and result ID from instruction desc table
if (spv::InstructionDesc[opCode].hasType()) { if (spv::InstructionDesc[opCode].hasType()) {
idFn(asId(word++)); idFn(asId(word++));
--numOperands; --numOperands;
} }
if (spv::InstructionDesc[opCode].hasResult()) { if (spv::InstructionDesc[opCode].hasResult()) {
idFn(asId(word++)); idFn(asId(word++));
--numOperands; --numOperands;
} }
// Extended instructions: currently, assume everything is an ID. // Extended instructions: currently, assume everything is an ID.
// TODO: add whatever data we need for exceptions to that // TODO: add whatever data we need for exceptions to that
if (opCode == spv::OpExtInst) { if (opCode == spv::OpExtInst) {
word += 2; // instruction set, and instruction from set word += 2; // instruction set, and instruction from set
numOperands -= 2; numOperands -= 2;
for (unsigned op=0; op < numOperands; ++op) for (unsigned op=0; op < numOperands; ++op)
idFn(asId(word++)); // ID idFn(asId(word++)); // ID
return nextInst; return nextInst;
} }
// Store IDs from instruction in our map // Store IDs from instruction in our map
for (int op = 0; op < spv::InstructionDesc[opCode].operands.getNum(); ++op, --numOperands) { for (int op = 0; op < spv::InstructionDesc[opCode].operands.getNum(); ++op, --numOperands) {
switch (spv::InstructionDesc[opCode].operands.getClass(op)) { switch (spv::InstructionDesc[opCode].operands.getClass(op)) {
case spv::OperandId: case spv::OperandId:
idFn(asId(word++)); idFn(asId(word++));
break; break;
case spv::OperandOptionalId: case spv::OperandOptionalId:
case spv::OperandVariableIds: case spv::OperandVariableIds:
for (unsigned i = 0; i < numOperands; ++i) for (unsigned i = 0; i < numOperands; ++i)
idFn(asId(word++)); idFn(asId(word++));
return nextInst; return nextInst;
case spv::OperandVariableLiterals: case spv::OperandVariableLiterals:
if (opCode == spv::OpDecorate && asDecoration(word - 1) == spv::DecorationBuiltIn) { if (opCode == spv::OpDecorate && asDecoration(word - 1) == spv::DecorationBuiltIn) {
++word; ++word;
--numOperands; --numOperands;
} }
word += numOperands; word += numOperands;
return nextInst; return nextInst;
case spv::OperandVariableLiteralId: case spv::OperandVariableLiteralId:
while (numOperands > 0) { while (numOperands > 0) {
++word; // immediate ++word; // immediate
idFn(asId(word++)); // ID idFn(asId(word++)); // ID
numOperands -= 2; numOperands -= 2;
} }
return nextInst; return nextInst;
case spv::OperandLiteralString: case spv::OperandLiteralString:
word += literalStringWords(literalString(word)); word += literalStringWords(literalString(word));
return nextInst; return nextInst;
// Single word operands we simply ignore, as they hold no IDs // Single word operands we simply ignore, as they hold no IDs
case spv::OperandLiteralNumber: case spv::OperandLiteralNumber:
case spv::OperandSource: case spv::OperandSource:
case spv::OperandExecutionModel: case spv::OperandExecutionModel:
case spv::OperandAddressing: case spv::OperandAddressing:
case spv::OperandMemory: case spv::OperandMemory:
case spv::OperandExecutionMode: case spv::OperandExecutionMode:
case spv::OperandStorage: case spv::OperandStorage:
case spv::OperandDimensionality: case spv::OperandDimensionality:
case spv::OperandDecoration: case spv::OperandDecoration:
case spv::OperandBuiltIn: case spv::OperandBuiltIn:
case spv::OperandSelect: case spv::OperandSelect:
case spv::OperandLoop: case spv::OperandLoop:
case spv::OperandFunction: case spv::OperandFunction:
case spv::OperandMemorySemantics: case spv::OperandMemorySemantics:
case spv::OperandMemoryAccess: case spv::OperandMemoryAccess:
case spv::OperandExecutionScope: case spv::OperandExecutionScope:
case spv::OperandGroupOperation: case spv::OperandGroupOperation:
case spv::OperandKernelEnqueueFlags: case spv::OperandKernelEnqueueFlags:
case spv::OperandKernelProfilingInfo: case spv::OperandKernelProfilingInfo:
++word; ++word;
break; break;
default: default:
break; break;
} }
} }
return nextInst; return nextInst;
} }
// Make a pass over all the instructions and process them given appropriate functions // Make a pass over all the instructions and process them given appropriate functions
spirvbin_t& spirvbin_t::process(instfn_t instFn, idfn_t idFn, int begin, int end) spirvbin_t& spirvbin_t::process(instfn_t instFn, idfn_t idFn, int begin, int end)
{ {
// For efficiency, reserve name map space. It can grow if needed. // For efficiency, reserve name map space. It can grow if needed.
nameMap.reserve(32); nameMap.reserve(32);
// If begin or end == 0, use defaults // If begin or end == 0, use defaults
begin = (begin == 0 ? header_size : begin); begin = (begin == 0 ? header_size : begin);
end = (end == 0 ? int(spv.size()) : end); end = (end == 0 ? int(spv.size()) : end);
// basic parsing and InstructionDesc table borrowed from SpvDisassemble.cpp... // basic parsing and InstructionDesc table borrowed from SpvDisassemble.cpp...
int nextInst = int(spv.size()); int nextInst = int(spv.size());
for (int word = begin; word < end; word = nextInst) for (int word = begin; word < end; word = nextInst)
nextInst = processInstruction(word, instFn, idFn); nextInst = processInstruction(word, instFn, idFn);
return *this; return *this;
} }
// Apply global name mapping to a single module // Apply global name mapping to a single module
void spirvbin_t::mapNames() void spirvbin_t::mapNames()
{ {
static const std::uint32_t softTypeIdLimit = 3011; // small prime. TODO: get from options static const std::uint32_t softTypeIdLimit = 3011; // small prime. TODO: get from options
static const std::uint32_t firstMappedID = 3019; // offset into ID space static const std::uint32_t firstMappedID = 3019; // offset into ID space
for (const auto& name : nameMap) { for (const auto& name : nameMap) {
std::uint32_t hashval = 1911; std::uint32_t hashval = 1911;
for (const char c : name.first) for (const char c : name.first)
hashval = hashval * 1009 + c; hashval = hashval * 1009 + c;
if (isOldIdUnmapped(name.second)) if (isOldIdUnmapped(name.second))
localId(name.second, nextUnusedId(hashval % softTypeIdLimit + firstMappedID)); localId(name.second, nextUnusedId(hashval % softTypeIdLimit + firstMappedID));
} }
} }
// Map fn contents to IDs of similar functions in other modules // Map fn contents to IDs of similar functions in other modules
void spirvbin_t::mapFnBodies() void spirvbin_t::mapFnBodies()
{ {
static const std::uint32_t softTypeIdLimit = 19071; // small prime. TODO: get from options static const std::uint32_t softTypeIdLimit = 19071; // small prime. TODO: get from options
static const std::uint32_t firstMappedID = 6203; // offset into ID space static const std::uint32_t firstMappedID = 6203; // offset into ID space
// Initial approach: go through some high priority opcodes first and assign them // Initial approach: go through some high priority opcodes first and assign them
// hash values. // hash values.
spv::Id fnId = spv::NoResult; spv::Id fnId = spv::NoResult;
std::vector<int> instPos; std::vector<int> instPos;
instPos.reserve(int(spv.size()) / 16); // initial estimate; can grow if needed. instPos.reserve(int(spv.size()) / 16); // initial estimate; can grow if needed.
// Build local table of instruction start positions // Build local table of instruction start positions
process( process(
[&](spv::Op, int start) { instPos.push_back(start); return true; }, [&](spv::Op, int start) { instPos.push_back(start); return true; },
op_fn_nop); op_fn_nop);
// Window size for context-sensitive canonicalization values // Window size for context-sensitive canonicalization values
// Emperical best size from a single data set. TODO: Would be a good tunable. // Emperical best size from a single data set. TODO: Would be a good tunable.
// We essentially performa a little convolution around each instruction, // We essentially performa a little convolution around each instruction,
// to capture the flavor of nearby code, to hopefully match to similar // to capture the flavor of nearby code, to hopefully match to similar
// code in other modules. // code in other modules.
static const int windowSize = 2; static const int windowSize = 2;
for (int entry = 0; entry < int(instPos.size()); ++entry) { for (int entry = 0; entry < int(instPos.size()); ++entry) {
const int start = instPos[entry]; const int start = instPos[entry];
const spv::Op opCode = asOpCode(start); const spv::Op opCode = asOpCode(start);
if (opCode == spv::OpFunction) if (opCode == spv::OpFunction)
fnId = asId(start + 2); fnId = asId(start + 2);
if (opCode == spv::OpFunctionEnd) if (opCode == spv::OpFunctionEnd)
fnId = spv::NoResult; fnId = spv::NoResult;
if (fnId != spv::NoResult) { // if inside a function if (fnId != spv::NoResult) { // if inside a function
const int word = start + (spv::InstructionDesc[opCode].hasType() ? 2 : 1); const int word = start + (spv::InstructionDesc[opCode].hasType() ? 2 : 1);
const int result = spv::InstructionDesc[opCode].hasResult() ? word : -1; const int result = spv::InstructionDesc[opCode].hasResult() ? word : -1;
if (result > 0) { if (result > 0) {
const spv::Id resId = asId(result); const spv::Id resId = asId(result);
std::uint32_t hashval = fnId * 17; // small prime std::uint32_t hashval = fnId * 17; // small prime
for (int i = entry-1; i >= entry-windowSize; --i) { for (int i = entry-1; i >= entry-windowSize; --i) {
if (asOpCode(instPos[i]) == spv::OpFunction) if (asOpCode(instPos[i]) == spv::OpFunction)
break; break;
hashval = hashval * 30103 + asOpCodeHash(instPos[i]); // 30103 = semiarbitrary prime hashval = hashval * 30103 + asOpCodeHash(instPos[i]); // 30103 = semiarbitrary prime
} }
for (int i = entry; i <= entry + windowSize; ++i) { for (int i = entry; i <= entry + windowSize; ++i) {
if (asOpCode(instPos[i]) == spv::OpFunctionEnd) if (asOpCode(instPos[i]) == spv::OpFunctionEnd)
break; break;
hashval = hashval * 30103 + asOpCodeHash(instPos[i]); // 30103 = semiarbitrary prime hashval = hashval * 30103 + asOpCodeHash(instPos[i]); // 30103 = semiarbitrary prime
} }
if (isOldIdUnmapped(resId)) if (isOldIdUnmapped(resId))
localId(resId, nextUnusedId(hashval % softTypeIdLimit + firstMappedID)); localId(resId, nextUnusedId(hashval % softTypeIdLimit + firstMappedID));
} }
} }
} }
spv::Op thisOpCode(spv::OpNop); spv::Op thisOpCode(spv::OpNop);
std::unordered_map<int, int> opCounter; std::unordered_map<int, int> opCounter;
int idCounter(0); int idCounter(0);
fnId = spv::NoResult; fnId = spv::NoResult;
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
switch (opCode) { switch (opCode) {
case spv::OpFunction: case spv::OpFunction:
// Reset counters at each function // Reset counters at each function
idCounter = 0; idCounter = 0;
opCounter.clear(); opCounter.clear();
fnId = asId(start + 2); fnId = asId(start + 2);
break; break;
case spv::OpTextureSample: case spv::OpTextureSample:
case spv::OpTextureSampleDref: case spv::OpTextureSampleDref:
case spv::OpTextureSampleLod: case spv::OpTextureSampleLod:
case spv::OpTextureSampleProj: case spv::OpTextureSampleProj:
case spv::OpTextureSampleGrad: case spv::OpTextureSampleGrad:
case spv::OpTextureSampleOffset: case spv::OpTextureSampleOffset:
case spv::OpTextureSampleProjLod: case spv::OpTextureSampleProjLod:
case spv::OpTextureSampleProjGrad: case spv::OpTextureSampleProjGrad:
case spv::OpTextureSampleLodOffset: case spv::OpTextureSampleLodOffset:
case spv::OpTextureSampleProjOffset: case spv::OpTextureSampleProjOffset:
case spv::OpTextureSampleGradOffset: case spv::OpTextureSampleGradOffset:
case spv::OpTextureSampleProjLodOffset: case spv::OpTextureSampleProjLodOffset:
case spv::OpTextureSampleProjGradOffset: case spv::OpTextureSampleProjGradOffset:
case spv::OpDot: case spv::OpDot:
case spv::OpCompositeExtract: case spv::OpCompositeExtract:
case spv::OpCompositeInsert: case spv::OpCompositeInsert:
case spv::OpVectorShuffle: case spv::OpVectorShuffle:
case spv::OpLabel: case spv::OpLabel:
case spv::OpVariable: case spv::OpVariable:
case spv::OpAccessChain: case spv::OpAccessChain:
case spv::OpLoad: case spv::OpLoad:
case spv::OpStore: case spv::OpStore:
case spv::OpCompositeConstruct: case spv::OpCompositeConstruct:
case spv::OpFunctionCall: case spv::OpFunctionCall:
++opCounter[opCode]; ++opCounter[opCode];
idCounter = 0; idCounter = 0;
thisOpCode = opCode; thisOpCode = opCode;
break; break;
default: default:
thisOpCode = spv::OpNop; thisOpCode = spv::OpNop;
} }
return false; return false;
}, },
[&](spv::Id& id) { [&](spv::Id& id) {
if (thisOpCode != spv::OpNop) { if (thisOpCode != spv::OpNop) {
++idCounter; ++idCounter;
const std::uint32_t hashval = opCounter[thisOpCode] * thisOpCode * 50047 + idCounter + fnId * 117; const std::uint32_t hashval = opCounter[thisOpCode] * thisOpCode * 50047 + idCounter + fnId * 117;
if (isOldIdUnmapped(id)) if (isOldIdUnmapped(id))
localId(id, nextUnusedId(hashval % softTypeIdLimit + firstMappedID)); localId(id, nextUnusedId(hashval % softTypeIdLimit + firstMappedID));
} }
}); });
} }
#ifdef NOTDEF #ifdef NOTDEF
// remove bodies of uncalled functions // remove bodies of uncalled functions
void spirvbin_t::offsetIds() void spirvbin_t::offsetIds()
{ {
// Count of how many functions each ID appears within // Count of how many functions each ID appears within
std::unordered_map<spv::Id, int> idFnCount; std::unordered_map<spv::Id, int> idFnCount;
std::unordered_map<spv::Id, int> idDefinedLoc; std::unordered_map<spv::Id, int> idDefinedLoc;
idset_t idsUsed; // IDs used in a given function idset_t idsUsed; // IDs used in a given function
int instCount = 0; int instCount = 0;
// create a count of how many functions each ID is used within // create a count of how many functions each ID is used within
process( process(
[&](spv::OpCode opCode, int start) { [&](spv::OpCode opCode, int start) {
++instCount; ++instCount;
switch (opCode) { switch (opCode) {
case spv::OpFunction: case spv::OpFunction:
for (const auto id : idsUsed) for (const auto id : idsUsed)
++idFnCount[id]; ++idFnCount[id];
idsUsed.clear(); idsUsed.clear();
break; break;
default: default:
{ {
const int word = start + (spv::InstructionDesc[opCode].hasType() ? 2 : 1); const int word = start + (spv::InstructionDesc[opCode].hasType() ? 2 : 1);
const int result = spv::InstructionDesc[opCode].hasResult() ? word : -1; const int result = spv::InstructionDesc[opCode].hasResult() ? word : -1;
if (result > 0) if (result > 0)
idDefinedLoc[asId(result)] = instCount; idDefinedLoc[asId(result)] = instCount;
} }
break; break;
} }
return false; return false;
}, },
[&](spv::Id& id) { idsUsed.insert(id); }); [&](spv::Id& id) { idsUsed.insert(id); });
// For each ID defined in exactly one function, replace uses by // For each ID defined in exactly one function, replace uses by
// negative offset to definitions in instructions. // negative offset to definitions in instructions.
static const int relOffsetLimit = 64; static const int relOffsetLimit = 64;
instCount = 0; instCount = 0;
process([&](spv::OpCode, int) { ++instCount; return false; }, process([&](spv::OpCode, int) { ++instCount; return false; },
[&](spv::Id& id) { [&](spv::Id& id) {
if (idFnCount[id] == 1 && (instCount - idDefinedLoc[id]) < relOffsetLimit) if (idFnCount[id] == 1 && (instCount - idDefinedLoc[id]) < relOffsetLimit)
id = idDefinedLoc[id] - instCount; id = idDefinedLoc[id] - instCount;
}); });
} }
#endif #endif
// EXPERIMENTAL: forward IO and uniform load/stores into operands // EXPERIMENTAL: forward IO and uniform load/stores into operands
// This produces invalid Schema-0 SPIRV // This produces invalid Schema-0 SPIRV
void spirvbin_t::forwardLoadStores() void spirvbin_t::forwardLoadStores()
{ {
idset_t fnLocalVars; // set of function local vars idset_t fnLocalVars; // set of function local vars
idmap_t idMap; // Map of load result IDs to what they load idmap_t idMap; // Map of load result IDs to what they load
// EXPERIMENTAL: Forward input and access chain loads into consumptions // EXPERIMENTAL: Forward input and access chain loads into consumptions
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
// Add inputs and uniforms to the map // Add inputs and uniforms to the map
if (((opCode == spv::OpVariable && asWordCount(start) == 4) || (opCode == spv::OpVariableArray)) && if (((opCode == spv::OpVariable && asWordCount(start) == 4) || (opCode == spv::OpVariableArray)) &&
(spv[start+3] == spv::StorageClassUniform || (spv[start+3] == spv::StorageClassUniform ||
spv[start+3] == spv::StorageClassUniformConstant || spv[start+3] == spv::StorageClassUniformConstant ||
spv[start+3] == spv::StorageClassInput)) spv[start+3] == spv::StorageClassInput))
fnLocalVars.insert(asId(start+2)); fnLocalVars.insert(asId(start+2));
if (opCode == spv::OpAccessChain && fnLocalVars.count(asId(start+3)) > 0) if (opCode == spv::OpAccessChain && fnLocalVars.count(asId(start+3)) > 0)
fnLocalVars.insert(asId(start+2)); fnLocalVars.insert(asId(start+2));
if (opCode == spv::OpLoad && fnLocalVars.count(asId(start+3)) > 0) { if (opCode == spv::OpLoad && fnLocalVars.count(asId(start+3)) > 0) {
idMap[asId(start+2)] = asId(start+3); idMap[asId(start+2)] = asId(start+3);
stripInst(start); stripInst(start);
} }
return false; return false;
}, },
[&](spv::Id& id) { if (idMap.find(id) != idMap.end()) id = idMap[id]; } [&](spv::Id& id) { if (idMap.find(id) != idMap.end()) id = idMap[id]; }
); );
// EXPERIMENTAL: Implicit output stores // EXPERIMENTAL: Implicit output stores
fnLocalVars.clear(); fnLocalVars.clear();
idMap.clear(); idMap.clear();
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
// Add inputs and uniforms to the map // Add inputs and uniforms to the map
if (((opCode == spv::OpVariable && asWordCount(start) == 4) || (opCode == spv::OpVariableArray)) && if (((opCode == spv::OpVariable && asWordCount(start) == 4) || (opCode == spv::OpVariableArray)) &&
(spv[start+3] == spv::StorageClassOutput)) (spv[start+3] == spv::StorageClassOutput))
fnLocalVars.insert(asId(start+2)); fnLocalVars.insert(asId(start+2));
if (opCode == spv::OpStore && fnLocalVars.count(asId(start+1)) > 0) { if (opCode == spv::OpStore && fnLocalVars.count(asId(start+1)) > 0) {
idMap[asId(start+2)] = asId(start+1); idMap[asId(start+2)] = asId(start+1);
stripInst(start); stripInst(start);
} }
return false; return false;
}, },
op_fn_nop); op_fn_nop);
process( process(
inst_fn_nop, inst_fn_nop,
[&](spv::Id& id) { if (idMap.find(id) != idMap.end()) id = idMap[id]; } [&](spv::Id& id) { if (idMap.find(id) != idMap.end()) id = idMap[id]; }
); );
strip(); // strip out data we decided to eliminate strip(); // strip out data we decided to eliminate
buildLocalMaps(); // rebuild ID mapping data buildLocalMaps(); // rebuild ID mapping data
} }
// remove bodies of uncalled functions // remove bodies of uncalled functions
void spirvbin_t::optLoadStore() void spirvbin_t::optLoadStore()
{ {
idset_t fnLocalVars; idset_t fnLocalVars;
// Map of load result IDs to what they load // Map of load result IDs to what they load
idmap_t idMap; idmap_t idMap;
// Find all the function local pointers stored at most once, and not via access chains // Find all the function local pointers stored at most once, and not via access chains
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
const int wordCount = asWordCount(start); const int wordCount = asWordCount(start);
// Add local variables to the map // Add local variables to the map
if ((opCode == spv::OpVariable && spv[start+3] == spv::StorageClassFunction && asWordCount(start) == 4) || if ((opCode == spv::OpVariable && spv[start+3] == spv::StorageClassFunction && asWordCount(start) == 4) ||
(opCode == spv::OpVariableArray && spv[start+3] == spv::StorageClassFunction)) (opCode == spv::OpVariableArray && spv[start+3] == spv::StorageClassFunction))
fnLocalVars.insert(asId(start+2)); fnLocalVars.insert(asId(start+2));
// Ignore process vars referenced via access chain // Ignore process vars referenced via access chain
if ((opCode == spv::OpAccessChain || opCode == spv::OpInBoundsAccessChain) && fnLocalVars.count(asId(start+3)) > 0) { if ((opCode == spv::OpAccessChain || opCode == spv::OpInBoundsAccessChain) && fnLocalVars.count(asId(start+3)) > 0) {
fnLocalVars.erase(asId(start+3)); fnLocalVars.erase(asId(start+3));
idMap.erase(asId(start+3)); idMap.erase(asId(start+3));
} }
if (opCode == spv::OpLoad && fnLocalVars.count(asId(start+3)) > 0) { if (opCode == spv::OpLoad && fnLocalVars.count(asId(start+3)) > 0) {
// Avoid loads before stores (TODO: why? Crashes driver, but seems like it shouldn't). // Avoid loads before stores (TODO: why? Crashes driver, but seems like it shouldn't).
if (idMap.find(asId(start+3)) == idMap.end()) { if (idMap.find(asId(start+3)) == idMap.end()) {
fnLocalVars.erase(asId(start+3)); fnLocalVars.erase(asId(start+3));
idMap.erase(asId(start+3)); idMap.erase(asId(start+3));
} }
// don't do for volatile references // don't do for volatile references
if (wordCount > 4 && (spv[start+4] & spv::MemoryAccessVolatileMask)) { if (wordCount > 4 && (spv[start+4] & spv::MemoryAccessVolatileMask)) {
fnLocalVars.erase(asId(start+3)); fnLocalVars.erase(asId(start+3));
idMap.erase(asId(start+3)); idMap.erase(asId(start+3));
} }
} }
if (opCode == spv::OpStore && fnLocalVars.count(asId(start+1)) > 0) { if (opCode == spv::OpStore && fnLocalVars.count(asId(start+1)) > 0) {
if (idMap.find(asId(start+1)) == idMap.end()) { if (idMap.find(asId(start+1)) == idMap.end()) {
idMap[asId(start+1)] = asId(start+2); idMap[asId(start+1)] = asId(start+2);
} else { } else {
// Remove if it has more than one store to the same pointer // Remove if it has more than one store to the same pointer
fnLocalVars.erase(asId(start+1)); fnLocalVars.erase(asId(start+1));
idMap.erase(asId(start+1)); idMap.erase(asId(start+1));
} }
// don't do for volatile references // don't do for volatile references
if (wordCount > 3 && (spv[start+3] & spv::MemoryAccessVolatileMask)) { if (wordCount > 3 && (spv[start+3] & spv::MemoryAccessVolatileMask)) {
fnLocalVars.erase(asId(start+3)); fnLocalVars.erase(asId(start+3));
idMap.erase(asId(start+3)); idMap.erase(asId(start+3));
} }
} }
return true; return true;
}, },
op_fn_nop); op_fn_nop);
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
if (opCode == spv::OpLoad && fnLocalVars.count(asId(start+3)) > 0) if (opCode == spv::OpLoad && fnLocalVars.count(asId(start+3)) > 0)
idMap[asId(start+2)] = idMap[asId(start+3)]; idMap[asId(start+2)] = idMap[asId(start+3)];
return false; return false;
}, },
op_fn_nop); op_fn_nop);
// Remove the load/store/variables for the ones we've discovered // Remove the load/store/variables for the ones we've discovered
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
if ((opCode == spv::OpLoad && fnLocalVars.count(asId(start+3)) > 0) || if ((opCode == spv::OpLoad && fnLocalVars.count(asId(start+3)) > 0) ||
(opCode == spv::OpStore && fnLocalVars.count(asId(start+1)) > 0) || (opCode == spv::OpStore && fnLocalVars.count(asId(start+1)) > 0) ||
(opCode == spv::OpVariable && fnLocalVars.count(asId(start+2)) > 0)) { (opCode == spv::OpVariable && fnLocalVars.count(asId(start+2)) > 0)) {
stripInst(start); stripInst(start);
return true; return true;
} }
return false; return false;
}, },
[&](spv::Id& id) { if (idMap.find(id) != idMap.end()) id = idMap[id]; } [&](spv::Id& id) { if (idMap.find(id) != idMap.end()) id = idMap[id]; }
); );
strip(); // strip out data we decided to eliminate strip(); // strip out data we decided to eliminate
buildLocalMaps(); // rebuild ID mapping data buildLocalMaps(); // rebuild ID mapping data
} }
// remove bodies of uncalled functions // remove bodies of uncalled functions
void spirvbin_t::dceFuncs() void spirvbin_t::dceFuncs()
{ {
msg(3, 2, std::string("Removing Dead Functions: ")); msg(3, 2, std::string("Removing Dead Functions: "));
// TODO: There are more efficient ways to do this. // TODO: There are more efficient ways to do this.
bool changed = true; bool changed = true;
while (changed) { while (changed) {
changed = false; changed = false;
for (auto fn = fnPos.begin(); fn != fnPos.end(); ) { for (auto fn = fnPos.begin(); fn != fnPos.end(); ) {
if (fn->first == entryPoint) { // don't DCE away the entry point! if (fn->first == entryPoint) { // don't DCE away the entry point!
++fn; ++fn;
continue; continue;
} }
const auto call_it = fnCalls.find(fn->first); const auto call_it = fnCalls.find(fn->first);
if (call_it == fnCalls.end() || call_it->second == 0) { if (call_it == fnCalls.end() || call_it->second == 0) {
changed = true; changed = true;
stripRange.push_back(fn->second); stripRange.push_back(fn->second);
fnPosDCE.insert(*fn); fnPosDCE.insert(*fn);
// decrease counts of called functions // decrease counts of called functions
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
if (opCode == spv::Op::OpFunctionCall) { if (opCode == spv::Op::OpFunctionCall) {
const auto call_it = fnCalls.find(asId(start + 3)); const auto call_it = fnCalls.find(asId(start + 3));
if (call_it != fnCalls.end()) { if (call_it != fnCalls.end()) {
if (--call_it->second <= 0) if (--call_it->second <= 0)
fnCalls.erase(call_it); fnCalls.erase(call_it);
} }
} }
return true; return true;
}, },
op_fn_nop, op_fn_nop,
fn->second.first, fn->second.first,
fn->second.second); fn->second.second);
fn = fnPos.erase(fn); fn = fnPos.erase(fn);
} else ++fn; } else ++fn;
} }
} }
} }
// remove unused function variables + decorations // remove unused function variables + decorations
void spirvbin_t::dceVars() void spirvbin_t::dceVars()
{ {
msg(3, 2, std::string("DCE Vars: ")); msg(3, 2, std::string("DCE Vars: "));
std::unordered_map<spv::Id, int> varUseCount; std::unordered_map<spv::Id, int> varUseCount;
// Count function variable use // Count function variable use
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
if (opCode == spv::OpVariable) { ++varUseCount[asId(start+2)]; return true; } if (opCode == spv::OpVariable) { ++varUseCount[asId(start+2)]; return true; }
return false; return false;
}, },
[&](spv::Id& id) { if (varUseCount[id]) ++varUseCount[id]; } [&](spv::Id& id) { if (varUseCount[id]) ++varUseCount[id]; }
); );
// Remove single-use function variables + associated decorations and names // Remove single-use function variables + associated decorations and names
process( process(
[&](spv::Op opCode, int start) { [&](spv::Op opCode, int start) {
if ((opCode == spv::OpVariable && varUseCount[asId(start+2)] == 1) || if ((opCode == spv::OpVariable && varUseCount[asId(start+2)] == 1) ||
(opCode == spv::OpDecorate && varUseCount[asId(start+1)] == 1) || (opCode == spv::OpDecorate && varUseCount[asId(start+1)] == 1) ||
(opCode == spv::OpName && varUseCount[asId(start+1)] == 1)) { (opCode == spv::OpName && varUseCount[asId(start+1)] == 1)) {
stripInst(start); stripInst(start);
} }
return true; return true;
}, },
op_fn_nop); op_fn_nop);
} }
// remove unused types // remove unused types
void spirvbin_t::dceTypes() void spirvbin_t::dceTypes()
{ {
std::vector<bool> isType(bound(), false); std::vector<bool> isType(bound(), false);
// for speed, make O(1) way to get to type query (map is log(n)) // for speed, make O(1) way to get to type query (map is log(n))
for (const auto typeStart : typeConstPos) for (const auto typeStart : typeConstPos)
isType[asTypeConstId(typeStart)] = true; isType[asTypeConstId(typeStart)] = true;
std::unordered_map<spv::Id, int> typeUseCount; std::unordered_map<spv::Id, int> typeUseCount;
// Count total type usage // Count total type usage
process(inst_fn_nop, process(inst_fn_nop,
[&](spv::Id& id) { if (isType[id]) ++typeUseCount[id]; } [&](spv::Id& id) { if (isType[id]) ++typeUseCount[id]; }
); );
// Remove types from deleted code // Remove types from deleted code
for (const auto& fn : fnPosDCE) for (const auto& fn : fnPosDCE)
process(inst_fn_nop, process(inst_fn_nop,
[&](spv::Id& id) { if (isType[id]) --typeUseCount[id]; }, [&](spv::Id& id) { if (isType[id]) --typeUseCount[id]; },
fn.second.first, fn.second.second); fn.second.first, fn.second.second);
// Remove single reference types // Remove single reference types
for (const auto typeStart : typeConstPos) { for (const auto typeStart : typeConstPos) {
const spv::Id typeId = asTypeConstId(typeStart); const spv::Id typeId = asTypeConstId(typeStart);
if (typeUseCount[typeId] == 1) { if (typeUseCount[typeId] == 1) {
--typeUseCount[typeId]; --typeUseCount[typeId];
stripInst(typeStart); stripInst(typeStart);
} }
} }
} }
#ifdef NOTDEF #ifdef NOTDEF
bool spirvbin_t::matchType(const spirvbin_t::globaltypes_t& globalTypes, spv::Id lt, spv::Id gt) const bool spirvbin_t::matchType(const spirvbin_t::globaltypes_t& globalTypes, spv::Id lt, spv::Id gt) const
{ {
// Find the local type id "lt" and global type id "gt" // Find the local type id "lt" and global type id "gt"
const auto lt_it = typeConstPosR.find(lt); const auto lt_it = typeConstPosR.find(lt);
if (lt_it == typeConstPosR.end()) if (lt_it == typeConstPosR.end())
return false; return false;
const auto typeStart = lt_it->second; const auto typeStart = lt_it->second;
// Search for entry in global table // Search for entry in global table
const auto gtype = globalTypes.find(gt); const auto gtype = globalTypes.find(gt);
if (gtype == globalTypes.end()) if (gtype == globalTypes.end())
return false; return false;
const auto& gdata = gtype->second; const auto& gdata = gtype->second;
// local wordcount and opcode // local wordcount and opcode
const int wordCount = asWordCount(typeStart); const int wordCount = asWordCount(typeStart);
const spv::Op opCode = asOpCode(typeStart); const spv::Op opCode = asOpCode(typeStart);
// no type match if opcodes don't match, or operand count doesn't match // no type match if opcodes don't match, or operand count doesn't match
if (opCode != opOpCode(gdata[0]) || wordCount != opWordCount(gdata[0])) if (opCode != opOpCode(gdata[0]) || wordCount != opWordCount(gdata[0]))
return false; return false;
const unsigned numOperands = wordCount - 2; // all types have a result const unsigned numOperands = wordCount - 2; // all types have a result
const auto cmpIdRange = [&](range_t range) { const auto cmpIdRange = [&](range_t range) {
for (int x=range.first; x<std::min(range.second, wordCount); ++x) for (int x=range.first; x<std::min(range.second, wordCount); ++x)
if (!matchType(globalTypes, asId(typeStart+x), gdata[x])) if (!matchType(globalTypes, asId(typeStart+x), gdata[x]))
return false; return false;
return true; return true;
}; };
const auto cmpConst = [&]() { return cmpIdRange(constRange(opCode)); }; const auto cmpConst = [&]() { return cmpIdRange(constRange(opCode)); };
const auto cmpSubType = [&]() { return cmpIdRange(typeRange(opCode)); }; const auto cmpSubType = [&]() { return cmpIdRange(typeRange(opCode)); };
// Compare literals in range [start,end) // Compare literals in range [start,end)
const auto cmpLiteral = [&]() { const auto cmpLiteral = [&]() {
const auto range = literalRange(opCode); const auto range = literalRange(opCode);
return std::equal(spir.begin() + typeStart + range.first, return std::equal(spir.begin() + typeStart + range.first,
spir.begin() + typeStart + std::min(range.second, wordCount), spir.begin() + typeStart + std::min(range.second, wordCount),
gdata.begin() + range.first); gdata.begin() + range.first);
}; };
assert(isTypeOp(opCode) || isConstOp(opCode)); assert(isTypeOp(opCode) || isConstOp(opCode));
switch (opCode) { switch (opCode) {
case spv::OpTypeOpaque: // TODO: disable until we compare the literal strings. case spv::OpTypeOpaque: // TODO: disable until we compare the literal strings.
case spv::OpTypeQueue: return false; case spv::OpTypeQueue: return false;
case spv::OpTypeEvent: // fall through... case spv::OpTypeEvent: // fall through...
case spv::OpTypeDeviceEvent: // ... case spv::OpTypeDeviceEvent: // ...
case spv::OpTypeReserveId: return false; case spv::OpTypeReserveId: return false;
// for samplers, we don't handle the optional parameters yet // for samplers, we don't handle the optional parameters yet
case spv::OpTypeSampler: return cmpLiteral() && cmpConst() && cmpSubType() && wordCount == 8; case spv::OpTypeSampler: return cmpLiteral() && cmpConst() && cmpSubType() && wordCount == 8;
default: return cmpLiteral() && cmpConst() && cmpSubType(); default: return cmpLiteral() && cmpConst() && cmpSubType();
} }
} }
// Look for an equivalent type in the globalTypes map // Look for an equivalent type in the globalTypes map
spv::Id spirvbin_t::findType(const spirvbin_t::globaltypes_t& globalTypes, spv::Id lt) const spv::Id spirvbin_t::findType(const spirvbin_t::globaltypes_t& globalTypes, spv::Id lt) const
{ {
// Try a recursive type match on each in turn, and return a match if we find one // Try a recursive type match on each in turn, and return a match if we find one
for (const auto& gt : globalTypes) for (const auto& gt : globalTypes)
if (matchType(globalTypes, lt, gt.first)) if (matchType(globalTypes, lt, gt.first))
return gt.first; return gt.first;
return spv::NoType; return spv::NoType;
} }
#endif // NOTDEF #endif // NOTDEF
// Return start position in SPV of given type. error if not found. // Return start position in SPV of given type. error if not found.
int spirvbin_t::typePos(spv::Id id) const int spirvbin_t::typePos(spv::Id id) const
{ {
const auto tid_it = typeConstPosR.find(id); const auto tid_it = typeConstPosR.find(id);
if (tid_it == typeConstPosR.end()) if (tid_it == typeConstPosR.end())
error("type ID not found"); error("type ID not found");
return tid_it->second; return tid_it->second;
} }
// Hash types to canonical values. This can return ID collisions (it's a bit // Hash types to canonical values. This can return ID collisions (it's a bit
// inevitable): it's up to the caller to handle that gracefully. // inevitable): it's up to the caller to handle that gracefully.
std::uint32_t spirvbin_t::hashType(int typeStart) const std::uint32_t spirvbin_t::hashType(int typeStart) const
{ {
const unsigned wordCount = asWordCount(typeStart); const unsigned wordCount = asWordCount(typeStart);
const spv::Op opCode = asOpCode(typeStart); const spv::Op opCode = asOpCode(typeStart);
switch (opCode) { switch (opCode) {
case spv::OpTypeVoid: return 0; case spv::OpTypeVoid: return 0;
case spv::OpTypeBool: return 1; case spv::OpTypeBool: return 1;
case spv::OpTypeInt: return 3 + (spv[typeStart+3]); case spv::OpTypeInt: return 3 + (spv[typeStart+3]);
case spv::OpTypeFloat: return 5; case spv::OpTypeFloat: return 5;
case spv::OpTypeVector: case spv::OpTypeVector:
return 6 + hashType(typePos(spv[typeStart+2])) * (spv[typeStart+3] - 1); return 6 + hashType(typePos(spv[typeStart+2])) * (spv[typeStart+3] - 1);
case spv::OpTypeMatrix: case spv::OpTypeMatrix:
return 30 + hashType(typePos(spv[typeStart+2])) * (spv[typeStart+3] - 1); return 30 + hashType(typePos(spv[typeStart+2])) * (spv[typeStart+3] - 1);
case spv::OpTypeSampler: case spv::OpTypeSampler:
return 120 + hashType(typePos(spv[typeStart+2])) + return 120 + hashType(typePos(spv[typeStart+2])) +
spv[typeStart+3] + // dimensionality spv[typeStart+3] + // dimensionality
spv[typeStart+4] * 8 * 16 + // content spv[typeStart+4] * 8 * 16 + // content
spv[typeStart+5] * 4 * 16 + // arrayed spv[typeStart+5] * 4 * 16 + // arrayed
spv[typeStart+6] * 2 * 16 + // compare spv[typeStart+6] * 2 * 16 + // compare
spv[typeStart+7] * 1 * 16; // multisampled spv[typeStart+7] * 1 * 16; // multisampled
case spv::OpTypeFilter: case spv::OpTypeFilter:
return 500; return 500;
case spv::OpTypeArray: case spv::OpTypeArray:
return 501 + hashType(typePos(spv[typeStart+2])) * spv[typeStart+3]; return 501 + hashType(typePos(spv[typeStart+2])) * spv[typeStart+3];
case spv::OpTypeRuntimeArray: case spv::OpTypeRuntimeArray:
return 5000 + hashType(typePos(spv[typeStart+2])); return 5000 + hashType(typePos(spv[typeStart+2]));
case spv::OpTypeStruct: case spv::OpTypeStruct:
{ {
std::uint32_t hash = 10000; std::uint32_t hash = 10000;
for (unsigned w=2; w < wordCount; ++w) for (unsigned w=2; w < wordCount; ++w)
hash += w * hashType(typePos(spv[typeStart+w])); hash += w * hashType(typePos(spv[typeStart+w]));
return hash; return hash;
} }
case spv::OpTypeOpaque: return 6000 + spv[typeStart+2]; case spv::OpTypeOpaque: return 6000 + spv[typeStart+2];
case spv::OpTypePointer: return 100000 + hashType(typePos(spv[typeStart+3])); case spv::OpTypePointer: return 100000 + hashType(typePos(spv[typeStart+3]));
case spv::OpTypeFunction: case spv::OpTypeFunction:
{ {
std::uint32_t hash = 200000; std::uint32_t hash = 200000;
for (unsigned w=2; w < wordCount; ++w) for (unsigned w=2; w < wordCount; ++w)
hash += w * hashType(typePos(spv[typeStart+w])); hash += w * hashType(typePos(spv[typeStart+w]));
return hash; return hash;
} }
case spv::OpTypeEvent: return 300000; case spv::OpTypeEvent: return 300000;
case spv::OpTypeDeviceEvent: return 300001; case spv::OpTypeDeviceEvent: return 300001;
case spv::OpTypeReserveId: return 300002; case spv::OpTypeReserveId: return 300002;
case spv::OpTypeQueue: return 300003; case spv::OpTypeQueue: return 300003;
case spv::OpTypePipe: return 300004; case spv::OpTypePipe: return 300004;
case spv::OpConstantNullObject: return 300005; case spv::OpConstantNullObject: return 300005;
case spv::OpConstantSampler: return 300006; case spv::OpConstantSampler: return 300006;
case spv::OpConstantTrue: return 300007; case spv::OpConstantTrue: return 300007;
case spv::OpConstantFalse: return 300008; case spv::OpConstantFalse: return 300008;
case spv::OpConstantNullPointer: return 300009; case spv::OpConstantNullPointer: return 300009;
case spv::OpConstantComposite: case spv::OpConstantComposite:
{ {
std::uint32_t hash = 300011 + hashType(typePos(spv[typeStart+1])); std::uint32_t hash = 300011 + hashType(typePos(spv[typeStart+1]));
for (unsigned w=3; w < wordCount; ++w) for (unsigned w=3; w < wordCount; ++w)
hash += w * hashType(typePos(spv[typeStart+w])); hash += w * hashType(typePos(spv[typeStart+w]));
return hash; return hash;
} }
case spv::OpConstant: case spv::OpConstant:
{ {
std::uint32_t hash = 400011 + hashType(typePos(spv[typeStart+1])); std::uint32_t hash = 400011 + hashType(typePos(spv[typeStart+1]));
for (unsigned w=3; w < wordCount; ++w) for (unsigned w=3; w < wordCount; ++w)
hash += w * spv[typeStart+w]; hash += w * spv[typeStart+w];
return hash; return hash;
} }
default: default:
error("unknown type opcode"); error("unknown type opcode");
return 0; return 0;
} }
} }
void spirvbin_t::mapTypeConst() void spirvbin_t::mapTypeConst()
{ {
globaltypes_t globalTypeMap; globaltypes_t globalTypeMap;
msg(3, 2, std::string("Remapping Consts & Types: ")); msg(3, 2, std::string("Remapping Consts & Types: "));
static const std::uint32_t softTypeIdLimit = 3011; // small prime. TODO: get from options static const std::uint32_t softTypeIdLimit = 3011; // small prime. TODO: get from options
static const std::uint32_t firstMappedID = 8; // offset into ID space static const std::uint32_t firstMappedID = 8; // offset into ID space
for (auto& typeStart : typeConstPos) { for (auto& typeStart : typeConstPos) {
const spv::Id resId = asTypeConstId(typeStart); const spv::Id resId = asTypeConstId(typeStart);
const std::uint32_t hashval = hashType(typeStart); const std::uint32_t hashval = hashType(typeStart);
if (isOldIdUnmapped(resId)) if (isOldIdUnmapped(resId))
localId(resId, nextUnusedId(hashval % softTypeIdLimit + firstMappedID)); localId(resId, nextUnusedId(hashval % softTypeIdLimit + firstMappedID));
} }
} }
// Strip a single binary by removing ranges given in stripRange // Strip a single binary by removing ranges given in stripRange
void spirvbin_t::strip() void spirvbin_t::strip()
{ {
if (stripRange.empty()) // nothing to do if (stripRange.empty()) // nothing to do
return; return;
// Sort strip ranges in order of traversal // Sort strip ranges in order of traversal
std::sort(stripRange.begin(), stripRange.end()); std::sort(stripRange.begin(), stripRange.end());
// Allocate a new binary big enough to hold old binary // Allocate a new binary big enough to hold old binary
// We'll step this iterator through the strip ranges as we go through the binary // We'll step this iterator through the strip ranges as we go through the binary
decltype(stripRange)::const_iterator strip_it = stripRange.begin(); decltype(stripRange)::const_iterator strip_it = stripRange.begin();
int strippedPos = 0; int strippedPos = 0;
for (unsigned word = 0; word < unsigned(spv.size()); ++word) { for (unsigned word = 0; word < unsigned(spv.size()); ++word) {
if (strip_it != stripRange.end() && word >= strip_it->second) if (strip_it != stripRange.end() && word >= strip_it->second)
++strip_it; ++strip_it;
if (strip_it == stripRange.end() || word < strip_it->first || word >= strip_it->second) if (strip_it == stripRange.end() || word < strip_it->first || word >= strip_it->second)
spv[strippedPos++] = spv[word]; spv[strippedPos++] = spv[word];
} }
spv.resize(strippedPos); spv.resize(strippedPos);
stripRange.clear(); stripRange.clear();
} }
// Strip a single binary by removing ranges given in stripRange // Strip a single binary by removing ranges given in stripRange
void spirvbin_t::remap(std::uint32_t opts) void spirvbin_t::remap(std::uint32_t opts)
{ {
options = opts; options = opts;
validate(); // validate header // Set up opcode tables from SpvDoc
buildLocalMaps(); spv::Parameterize();
msg(3, 4, std::string("ID bound: ") + std::to_string(bound())); validate(); // validate header
buildLocalMaps();
if (options & OPT_LOADSTORE) optLoadStore();
if (options & OPT_FWD_LS) forwardLoadStores(); msg(3, 4, std::string("ID bound: ") + std::to_string(bound()));
if (options & DCE_FUNCS) dceFuncs();
if (options & DCE_VARS) dceVars(); if (options & OPT_LOADSTORE) optLoadStore();
if (options & DCE_TYPES) dceTypes(); if (options & OPT_FWD_LS) forwardLoadStores();
if (options & MAP_TYPES) mapTypeConst(); if (options & DCE_FUNCS) dceFuncs();
if (options & MAP_NAMES) mapNames(); if (options & DCE_VARS) dceVars();
if (options & MAP_FUNCS) mapFnBodies(); if (options & DCE_TYPES) dceTypes();
// if (options & STRIP) stripDebug(); if (options & MAP_TYPES) mapTypeConst();
if (options & MAP_NAMES) mapNames();
mapRemainder(); // map any unmapped IDs if (options & MAP_FUNCS) mapFnBodies();
applyMap(); // Now remap each shader to the new IDs we've come up with // if (options & STRIP) stripDebug();
strip(); // strip out data we decided to eliminate
mapRemainder(); // map any unmapped IDs
#define EXPERIMENT3 0 applyMap(); // Now remap each shader to the new IDs we've come up with
#if (EXPERIMENT3) strip(); // strip out data we decided to eliminate
// TODO: ... shortcuts for simple single-const access chains and constants,
// folded into high half of the ID space. #define EXPERIMENT3 0
#endif #if (EXPERIMENT3)
} // TODO: ... shortcuts for simple single-const access chains and constants,
// folded into high half of the ID space.
// remap from a memory image #endif
void spirvbin_t::remap(std::vector<std::uint32_t>& in_spv, std::uint32_t opts) }
{
spv.swap(in_spv); // remap from a memory image
remap(opts); void spirvbin_t::remap(std::vector<std::uint32_t>& in_spv, std::uint32_t opts)
spv.swap(in_spv); {
} spv.swap(in_spv);
remap(opts);
spv.swap(in_spv);
}
} // namespace SPV } // namespace SPV
#endif // defined (use_cpp11) #endif // defined (use_cpp11)
StandAlone/CMakeLists.txt
...@@ -10,8 +10,10 @@ else(WIN32) ...@@ -10,8 +10,10 @@ else(WIN32)
endif(WIN32) endif(WIN32)
set(SOURCES StandAlone.cpp) set(SOURCES StandAlone.cpp)
set(REMAPPER_SOURCES spirv-remap.cpp)
add_executable(glslangValidator ${SOURCES}) add_executable(glslangValidator ${SOURCES})
add_executable(spirv-remap ${REMAPPER_SOURCES})
set(LIBRARIES set(LIBRARIES
glslang glslang
...@@ -26,6 +28,7 @@ elseif(UNIX) ...@@ -26,6 +28,7 @@ elseif(UNIX)
endif(WIN32) endif(WIN32)
target_link_libraries(glslangValidator ${LIBRARIES}) target_link_libraries(glslangValidator ${LIBRARIES})
target_link_libraries(spirv-remap ${LIBRARIES})
if(WIN32) if(WIN32)
source_group("Source" FILES ${SOURCES}) source_group("Source" FILES ${SOURCES})
...@@ -33,3 +36,6 @@ endif(WIN32) ...@@ -33,3 +36,6 @@ endif(WIN32)
install(TARGETS glslangValidator install(TARGETS glslangValidator
RUNTIME DESTINATION bin) RUNTIME DESTINATION bin)
install(TARGETS spirv-remap
RUNTIME DESTINATION bin)
StandAlone/spirv-remap.cpp 0 → 100644
//
//Copyright (C) 2015 LunarG, Inc.
//
//All rights reserved.
//
//Redistribution and use in source and binary forms, with or without
//modification, are permitted provided that the following conditions
//are met:
//
// Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
//
// Redistributions in binary form must reproduce the above
// copyright notice, this list of conditions and the following
// disclaimer in the documentation and/or other materials provided
// with the distribution.
//
// Neither the name of 3Dlabs Inc. Ltd. nor the names of its
// contributors may be used to endorse or promote products derived
// from this software without specific prior written permission.
//
//THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
//"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
//LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
//FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
//COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
//INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
//BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
//LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
//CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
//LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
//ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
//POSSIBILITY OF SUCH DAMAGE.
//
#include <iostream>
#include <fstream>
#include <cstring>
#include <stdexcept>
#include "../SPIRV/SPVRemapper.h"
namespace {
typedef unsigned int SpvWord;
// Poor man's basename: given a complete path, return file portion.
// E.g:
// Linux: /foo/bar/test -> test
// Win: c:\foo\bar\test -> test
// It's not very efficient, but that doesn't matter for our minimal-duty use.
// Using boost::filesystem would be better in many ways, but want to avoid that dependency.
// OS dependent path separator (avoiding boost::filesystem dependency)
#if defined(_WIN32)
char path_sep_char() { return '\\'; }
#else
char path_sep_char() { return '/'; }
#endif
std::string basename(const std::string filename)
{
const size_t sepLoc = filename.find_last_of(path_sep_char());
return (sepLoc == filename.npos) ? filename : filename.substr(sepLoc+1);
}
void errHandler(const std::string& str) {
std::cout << str << std::endl;
exit(5);
}
void logHandler(const std::string& str) {
std::cout << str << std::endl;
}
// Read word stream from disk
void read(std::vector<SpvWord>& spv, const std::string& inFilename)
{
std::ifstream fp;
std::cout << " reading: " << inFilename << std::endl;
spv.clear();
fp.open(inFilename, std::fstream::in | std::fstream::binary);
if (fp.fail())
errHandler("error opening file for read: ");
// Reserve space (for efficiency, not for correctness)
fp.seekg(0, fp.end);
spv.reserve(size_t(fp.tellg()) / sizeof(SpvWord));
fp.seekg(0, fp.beg);
while (!fp.eof()) {
SpvWord inWord;
fp.read((char *)&inWord, sizeof(inWord));
if (!fp.eof()) {
spv.push_back(inWord);
if (fp.fail())
errHandler(std::string("error reading file: ") + inFilename);
}
}
}
void write(std::vector<SpvWord>& spv, const std::string& outFile)
{
if (outFile.empty())
errHandler("missing output filename.");
std::ofstream fp;
std::cout << " writing: " << outFile << std::endl;
fp.open(outFile, std::fstream::out | std::fstream::binary);
if (fp.fail())
errHandler(std::string("error opening file for write: ") + outFile);
for (auto word : spv) {
fp.write((char *)&word, sizeof(word));
if (fp.fail())
errHandler(std::string("error writing file: ") + outFile);
}
// file is closed by destructor
}
// Print helpful usage message to stdout, and exit
void usage(const char* const name, const char* const msg = 0)
{
if (msg)
std::cout << msg << std::endl << std::endl;
std::cout << "Usage: " << std::endl;
std::cout << " " << basename(name)
<< " [-v[v[...]] | --verbose [int]]"
<< " [--map (all|types|names|funcs)]"
<< " [--dce (all|types|funcs)]"
<< " [--opt (all|loadstore)]"
<< " [--strip-all | --strip all | -s]"
<< " [--do-everything]"
<< " --input | -i file1 [file2...] --output|-o DESTDIR"
<< std::endl;
std::cout << " " << basename(name) << " [--version | -V]" << std::endl;
std::cout << " " << basename(name) << " [--help | -?]" << std::endl;
exit(5);
}
// grind through each SPIR in turn
void execute(const std::vector<std::string>& inputFile, const std::string& outputDir,
int opts, int verbosity)
{
for (const auto& filename : inputFile) {
std::vector<SpvWord> spv;
read(spv, filename);
spv::spirvbin_t(verbosity).remap(spv, opts);
const std::string outfile = outputDir + path_sep_char() + basename(filename);
write(spv, outfile);
}
if (verbosity > 0)
std::cout << "Done: " << inputFile.size() << " file(s) processed" << std::endl;
}
// Parse command line options
void parseCmdLine(int argc, char** argv, std::vector<std::string>& inputFile,
std::string& outputDir,
int& options,
int& verbosity)
{
if (argc < 2)
usage(argv[0]);
verbosity = 0;
options = spv::spirvbin_t::Options::NONE;
// Parse command line.
// boost::program_options would be quite a bit nicer, but we don't want to
// introduce a dependency on boost.
for (int a=1; a<argc; ) {
const std::string arg = argv[a];
if (arg == "--output" || arg == "-o") {
// Output directory
if (++a >= argc)
usage(argv[0], "--output requires an argument");
if (!outputDir.empty())
usage(argv[0], "--output can be provided only once");
outputDir = argv[a++];
// Remove trailing directory separator characters
while (!outputDir.empty() && outputDir.back() == path_sep_char())
outputDir.pop_back();
}
else if (arg == "-vv") { verbosity = 2; ++a; } // verbosity shortcuts
else if (arg == "-vvv") { verbosity = 3; ++a; } // ...
else if (arg == "-vvvv") { verbosity = 4; ++a; } // ...
else if (arg == "-vvvvv") { verbosity = 5; ++a; } // ...
else if (arg == "--verbose" || arg == "-v") {
++a;
verbosity = 1;
if (a < argc) {
try {
verbosity = std::stoi(argv[a]);
++a;
} catch (const std::invalid_argument&) { } // ok to have no numeric value
}
}
else if (arg == "--version" || arg == "-V") {
std::cout << basename(argv[0]) << " version 0.97 " << __DATE__ << " " << __TIME__ << std::endl;
exit(0);
} else if (arg == "--input" || arg == "-i") {
// Collect input files
for (++a; a < argc && argv[a][0] != '-'; ++a)
inputFile.push_back(argv[a]);
} else if (arg == "--do-everything") {
++a;
options = options | spv::spirvbin_t::Options::DO_EVERYTHING;
} else if (arg == "--strip-all" || arg == "-s") {
++a;
options = options | spv::spirvbin_t::Options::STRIP;
} else if (arg == "--strip") {
++a;
if (strncmp(argv[a], "all", 3) == 0) {
options = options | spv::spirvbin_t::Options::STRIP;
++a;
}
} else if (arg == "--dce") {
// Parse comma (or colon, etc) separated list of things to dce
++a;
for (const char* c = argv[a]; *c; ++c) {
if (strncmp(c, "all", 3) == 0) {
options = (options | spv::spirvbin_t::Options::DCE_ALL);
c += 3;
} else if (strncmp(c, "*", 1) == 0) {
options = (options | spv::spirvbin_t::Options::DCE_ALL);
c += 1;
} else if (strncmp(c, "funcs", 5) == 0) {
options = (options | spv::spirvbin_t::Options::DCE_FUNCS);
c += 5;
} else if (strncmp(c, "types", 5) == 0) {
options = (options | spv::spirvbin_t::Options::DCE_TYPES);
c += 5;
}
}
++a;
} else if (arg == "--map") {
// Parse comma (or colon, etc) separated list of things to map
++a;
for (const char* c = argv[a]; *c; ++c) {
if (strncmp(c, "all", 3) == 0) {
options = (options | spv::spirvbin_t::Options::MAP_ALL);
c += 3;
} else if (strncmp(c, "*", 1) == 0) {
options = (options | spv::spirvbin_t::Options::MAP_ALL);
c += 1;
} else if (strncmp(c, "types", 5) == 0) {
options = (options | spv::spirvbin_t::Options::MAP_TYPES);
c += 5;
} else if (strncmp(c, "names", 5) == 0) {
options = (options | spv::spirvbin_t::Options::MAP_NAMES);
c += 5;
} else if (strncmp(c, "funcs", 5) == 0) {
options = (options | spv::spirvbin_t::Options::MAP_FUNCS);
c += 5;
}
}
++a;
} else if (arg == "--opt") {
++a;
for (const char* c = argv[a]; *c; ++c) {
if (strncmp(c, "all", 3) == 0) {
options = (options | spv::spirvbin_t::Options::OPT_ALL);
c += 3;
} else if (strncmp(c, "*", 1) == 0) {
options = (options | spv::spirvbin_t::Options::OPT_ALL);
c += 1;
} else if (strncmp(c, "loadstore", 9) == 0) {
options = (options | spv::spirvbin_t::Options::OPT_LOADSTORE);
c += 9;
}
}
++a;
} else if (arg == "--help" || arg == "-?") {
usage(argv[0]);
} else {
usage(argv[0], "Unknown command line option");
}
}
}
} // namespace
int main(int argc, char** argv)
{
#ifdef use_cpp11
std::vector<std::string> inputFile;
std::string outputDir;
int opts;
int verbosity;
// handle errors by exiting
spv::spirvbin_t::registerErrorHandler(errHandler);
// Log messages to std::cout
spv::spirvbin_t::registerLogHandler(logHandler);
if (argc < 2)
usage(argv[0]);
parseCmdLine(argc, argv, inputFile, outputDir, opts, verbosity);
if (outputDir.empty())
usage(argv[0], "Output directory required");
std::string errmsg;
// Main operations: read, remap, and write.
execute(inputFile, outputDir, opts, verbosity);
#endif
// If we get here, everything went OK! Nothing more to be done.
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment