Commit f251995d by Jamie Madill Committed by Commit Bot

Capture/Replay: Write capture index file.

This file will be used with multi-frame captures to share common code. Common code is global state, resource maps, and a list of frame replay functions. This should make converting a CPP replay into a functional test quite a bit simpler. The replay files will now be something like: angle_capture_context1.cpp angle_capture_context1.h angle_capture_context1_frame000.cpp angle_capture_context1_frame001.cpp ... etc Also adds a template for adding a capture/replay sample. Instructions are located in samples/BUILD.gn and docs in doc/CaptureAndReplay.md. Bug: angleproject:3611 Change-Id: I437b338fd84689d670a7d9e3e219d9334de25fd8 Reviewed-on: https://chromium-review.googlesource.com/c/angle/angle/+/1869543 Commit-Queue: Jamie Madill <jmadill@chromium.org> Reviewed-by: 's avatarTobin Ehlis <tobine@google.com> Reviewed-by: 's avatarCody Northrop <cnorthrop@google.com>
parent 7af2676b
......@@ -18,14 +18,10 @@ To build ANGLE with capture and replay enabled update your GN args:
angle_with_capture_by_default = true
```
Once built ANGLE will capture a frame's OpenGL calls to a CPP replay stored in the current working
directory. The capture files will be named `angle_capture_context{id}_frame{n}.cpp`. Each OpenGL
context has a unique Context ID to identify its proper replay files. ANGLE will write out large
binary blobs such as Texture or Buffer data to `angle_capture_context{id}_frame{n}.angledata`.
To run a CPP replay you must stitch together the source files into a GN target. The samples
framework is well-suited for implementing a CPP replay example. Alternately you could adapt an ANGLE
end-to-end test. The `angledata` files must be accessible to the replay.
Once built ANGLE will capture the OpenGL ES calls to a CPP replay. By default the replay will be
stored in the current working directory. The capture files will be named according to the pattern
`angle_capture_context{id}_frame{n}.cpp`. ANGLE will additionally write out data binary blobs for
Texture or Buffer contexts to `angle_capture_context{id}_frame{n}.angledata`.
## Controlling Frame Capture
......@@ -39,3 +35,35 @@ Some simple environment variables control frame capture:
* `ANGLE_CAPTURE_FRAME_END=<n>`:
By default ANGLE will capture the first ten frames. This variable can override the default.
Example: `ANGLE_CAPTURE_FRAME_END=4`
A good way to test out the capture is to use environment variables in conjunction with the sample
template. For example:
```
$ ANGLE_CAPTURE_FRAME_END=4 ANGLE_CAPTURE_OUT_DIR=samples/capture_replay out/Debug/simple_texture_2d
```
## Running a CPP replay
To run a CPP replay you can use a template located in
[samples/capture_and_replay](../samples/capture_and_replay). Update
[samples/BUILD.gn](../samples/BUILD.gn) to enable the `capture_replay` sample to include your replay:
```
capture_replay("my_sample") {
sources = [
"capture_replay/angle_capture_context1_frame000.cpp",
"capture_replay/angle_capture_context1_frame001.cpp",
"capture_replay/angle_capture_context1_frame002.cpp",
"capture_replay/angle_capture_context1_frame003.cpp",
"capture_replay/angle_capture_context1_frame004.cpp",
]
}
```
Then build and run your replay sample:
```
$ autoninja -C out/Debug my_sample
$ ANGLE_CAPTURE_ENABLED=0 out/Debug/my_sample
```
......@@ -55,13 +55,21 @@ template("angle_sample") {
}
angle_executable(target_name) {
forward_variables_from(invoker,
[
"sources",
"cflags",
])
deps = [
":sample_util",
]
if (defined(invoker.data)) {
deps += [ ":${target_name}_data" ]
}
sources = invoker.sources
if (defined(invoker.suppressed_configs)) {
suppressed_configs += invoker.suppressed_configs
}
}
}
......@@ -209,6 +217,31 @@ angle_sample("gles1_draw_texture") {
]
}
template("capture_replay") {
angle_sample(target_name) {
sources = invoker.sources + [
"capture_replay/CaptureReplay.cpp",
"capture_replay/angle_capture_context1.cpp",
"capture_replay/angle_capture_context1.h",
]
suppressed_configs = [ "$angle_root:constructor_and_destructor_warnings" ]
}
}
# The capture_replay sample is set up to work with a single Context.
# To use the capture replay sample fist move your capture sources into
# the capture_replay folder. Uncomment and update this target:
# capture_replay("my_sample") {
# sources = [
# "capture_replay/angle_capture_context1_frame000.cpp",
# "capture_replay/angle_capture_context1_frame001.cpp",
# "capture_replay/angle_capture_context1_frame002.cpp",
# "capture_replay/angle_capture_context1_frame003.cpp",
# "capture_replay/angle_capture_context1_frame004.cpp",
# ]
# }
group("all") {
testonly = true
deps = [
......
angle_capture_context*
\ No newline at end of file
//
// Copyright 2019 The ANGLE Project Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
//
// CaptureReplay: Template for replaying a frame capture with ANGLE.
#include "SampleApplication.h"
#include "angle_capture_context1.h"
class CaptureReplaySample : public SampleApplication
{
public:
CaptureReplaySample(int argc, char **argv)
: SampleApplication("CaptureReplaySample", argc, argv, 2, 0)
{}
bool initialize() override { return true; }
void destroy() override {}
void draw() override
{
ReplayContext1Frame(kReplayFrameStart + (mCurrentFrame % kReplayFrameEnd));
mCurrentFrame++;
}
private:
uint32_t mCurrentFrame = 0;
};
int main(int argc, char **argv)
{
CaptureReplaySample app(argc, argv);
return app.run();
}
......@@ -11,6 +11,7 @@
#include <cerrno>
#include <cstring>
#include <fstream>
#include <string>
#include "common/system_utils.h"
......@@ -61,11 +62,38 @@ constexpr char kOutDirectoryVarName[] = "ANGLE_CAPTURE_OUT_DIR";
constexpr char kFrameStartVarName[] = "ANGLE_CAPTURE_FRAME_START";
constexpr char kFrameEndVarName[] = "ANGLE_CAPTURE_FRAME_END";
struct FmtCapturePrefix
{
FmtCapturePrefix(int contextIdIn) : contextId(contextIdIn) {}
int contextId;
};
std::ostream &operator<<(std::ostream &os, const FmtCapturePrefix &fmt)
{
os << "angle_capture_context" << fmt.contextId;
return os;
}
struct FmtReplayFunction
{
FmtReplayFunction(int contextIdIn, uint32_t frameIndexIn)
: contextId(contextIdIn), frameIndex(frameIndexIn)
{}
int contextId;
uint32_t frameIndex;
};
std::ostream &operator<<(std::ostream &os, const FmtReplayFunction &fmt)
{
os << "ReplayContext" << fmt.contextId << "Frame" << fmt.frameIndex << "()";
return os;
}
std::string GetCaptureFileName(int contextId, uint32_t frameIndex, const char *suffix)
{
std::stringstream fnameStream;
fnameStream << "angle_capture_context" << contextId << "_frame" << std::setfill('0')
<< std::setw(3) << frameIndex << suffix;
fnameStream << FmtCapturePrefix(contextId) << "_frame" << std::setfill('0') << std::setw(3)
<< frameIndex << suffix;
return fnameStream.str();
}
......@@ -266,12 +294,11 @@ void WriteCppReplayForCall(const CallCapture &call,
if (param.arrayClientPointerIndex != -1)
{
callOut << "gClientArrays[" << param.arrayClientPointerIndex << "].data()";
callOut << "gClientArrays[" << param.arrayClientPointerIndex << "]";
}
else if (param.readBufferSizeBytes > 0)
{
callOut << "reinterpret_cast<" << ParamTypeToString(param.type)
<< ">(gReadBuffer.data())";
callOut << "reinterpret_cast<" << ParamTypeToString(param.type) << ">(gReadBuffer)";
}
else if (param.data.empty())
{
......@@ -360,166 +387,263 @@ void WriteCppReplayForCall(const CallCapture &call,
out << callOut.str();
}
bool AnyClientArray(const gl::AttribArray<size_t> &clientArraySizes)
size_t MaxClientArraySize(const gl::AttribArray<size_t> &clientArraySizes)
{
size_t found = 0;
for (size_t size : clientArraySizes)
{
if (size > 0)
return true;
if (size > found)
found = size;
}
return false;
return found;
}
void WriteCppReplay(const std::string &outDir,
int contextId,
uint32_t frameIndex,
const std::vector<CallCapture> &calls,
const gl::AttribArray<size_t> &clientArraySizes,
size_t readBufferSize)
struct SaveFileHelper
{
bool useClientArrays = AnyClientArray(clientArraySizes);
SaveFileHelper(const std::string &filePathIn, std::ios_base::openmode mode = std::ios::out)
: ofs(filePathIn, mode), filePath(filePathIn)
{
if (!ofs.is_open())
{
FATAL() << "Could not open " << filePathIn;
}
}
// Count resource IDs.
angle::PackedEnumMap<ResourceIDType, uint32_t, angle::kParamTypeCount> resourceIDCounts = {};
for (const CallCapture &call : calls)
~SaveFileHelper() { printf("Saved '%s'.\n", filePath.c_str()); }
template <typename T>
SaveFileHelper &operator<<(const T &value)
{
for (const ParamCapture &param : call.params.getParamCaptures())
ofs << value;
if (ofs.bad())
{
ResourceIDType idType = GetResourceIDTypeFromParamType(param.type);
if (idType != ResourceIDType::InvalidEnum)
{
resourceIDCounts[idType]++;
}
FATAL() << "Error writing to " << filePath;
}
return *this;
}
std::ofstream ofs;
std::string filePath;
};
void WriteCppReplay(const std::string &outDir,
int contextId,
uint32_t frameIndex,
const std::vector<CallCapture> &calls)
{
DataCounters counters;
std::stringstream out;
std::stringstream header;
std::vector<uint8_t> binaryData;
header << "#include \"util/gles_loader_autogen.h\"\n";
header << "\n";
header << "#include <cstdio>\n";
header << "#include <cstring>\n";
header << "#include <vector>\n";
header << "#include <unordered_map>\n";
header << "#include \"" << FmtCapturePrefix(contextId) << ".h\"\n";
header << "";
header << "\n";
header << "namespace\n";
header << "{\n";
if (readBufferSize > 0)
out << "void " << FmtReplayFunction(contextId, frameIndex) << "\n";
out << "{\n";
if (!binaryData.empty())
{
header << "std::vector<uint8_t> gReadBuffer;\n";
std::string binaryDataFileName = GetCaptureFileName(contextId, frameIndex, ".angledata");
out << " LoadBinaryData(\"" << binaryDataFileName << "\", "
<< static_cast<int>(binaryData.size()) << ");\n";
}
if (useClientArrays)
for (const CallCapture &call : calls)
{
header << "std::vector<uint8_t> gClientArrays[" << gl::MAX_VERTEX_ATTRIBS << "];\n";
header << "void UpdateClientArrayPointer(int arrayIndex, const void *data, GLuint64 size)"
<< "\n";
header << "{\n";
header << " memcpy(gClientArrays[arrayIndex].data(), data, size);\n";
header << "}\n";
out << " ";
WriteCppReplayForCall(call, &counters, out, header, &binaryData);
out << ";\n";
}
header << "using ResourceMap = std::unordered_map<GLuint, GLuint>;\n";
header << "void UpdateResourceMap(ResourceMap *resourceMap, GLuint id, GLsizei "
"readBufferOffset)\n";
header << "{\n";
header << " GLuint returnedID;\n";
header << " memcpy(&returnedID, &gReadBuffer[readBufferOffset], sizeof(GLuint));\n";
header << " (*resourceMap)[id] = returnedID;\n";
header << "}\n";
header << "\n";
header << "// Resource Maps\n";
if (!binaryData.empty())
{
std::string dataFilepath = GetCaptureFilePath(outDir, contextId, frameIndex, ".angledata");
SaveFileHelper saveData(dataFilepath, std::ios::binary);
saveData.ofs.write(reinterpret_cast<const char *>(binaryData.data()), binaryData.size());
}
out << "}\n";
header << "} // anonymous namespace\n";
for (ResourceIDType resourceType : AllEnums<ResourceIDType>())
{
if (resourceIDCounts[resourceType] == 0)
continue;
std::string outString = out.str();
std::string headerString = header.str();
const char *name = GetResourceIDTypeName(resourceType);
header << "ResourceMap g" << name << "Map;\n";
header << "void Update" << name << "ID(GLuint id, GLsizei readBufferOffset)\n";
header << "{\n";
header << " UpdateResourceMap(&g" << name << "Map, id, readBufferOffset);\n";
header << "}\n";
std::string cppFilePath = GetCaptureFilePath(outDir, contextId, frameIndex, ".cpp");
SaveFileHelper saveCpp(cppFilePath);
saveCpp << headerString << "\n\n" << outString;
}
}
out << "void ReplayFrame" << frameIndex << "()\n";
out << "{\n";
out << " LoadBinaryData();\n";
void WriteCppReplayIndexFiles(const std::string &outDir,
int contextId,
uint32_t frameStart,
uint32_t frameEnd,
size_t readBufferSize,
const gl::AttribArray<size_t> &clientArraySizes,
const HasResourceTypeMap &hasResourceType)
{
size_t maxClientArraySize = MaxClientArraySize(clientArraySizes);
std::stringstream header;
std::stringstream source;
for (size_t arrayIndex = 0; arrayIndex < clientArraySizes.size(); ++arrayIndex)
header << "#pragma once\n";
header << "\n";
header << "#include \"util/gles_loader_autogen.h\"\n";
header << "\n";
header << "#include <cstdint>\n";
header << "#include <cstdio>\n";
header << "#include <cstring>\n";
header << "#include <unordered_map>\n";
header << "\n";
header << "// Replay functions\n";
header << "\n";
header << "constexpr uint32_t kReplayFrameStart = " << frameStart << ";\n";
header << "constexpr uint32_t kReplayFrameEnd = " << frameEnd << ";\n";
header << "\n";
header << "void ReplayContext" << contextId << "Frame(uint32_t frameIndex);\n";
header << "\n";
for (uint32_t frameIndex = frameStart; frameIndex < frameEnd; ++frameIndex)
{
if (clientArraySizes[arrayIndex] > 0)
{
out << " gClientArrays[" << arrayIndex << "].resize(" << clientArraySizes[arrayIndex]
<< ");\n";
}
header << "void " << FmtReplayFunction(contextId, frameIndex) << ";\n";
}
header << "\n";
header << "void LoadBinaryData(const char *fileName, size_t size);\n";
header << "\n";
header << "// Global state\n";
header << "\n";
header << "using ResourceMap = std::unordered_map<GLuint, GLuint>;\n";
header << "\n";
header << "extern uint8_t *gBinaryData;\n";
source << "#include \"" << FmtCapturePrefix(contextId) << ".h\"\n";
source << "\n";
source << "namespace\n";
source << "{\n";
source << "void UpdateResourceMap(ResourceMap *resourceMap, GLuint id, GLsizei "
"readBufferOffset)\n";
source << "{\n";
source << " GLuint returnedID;\n";
source << " memcpy(&returnedID, &gReadBuffer[readBufferOffset], sizeof(GLuint));\n";
source << " (*resourceMap)[id] = returnedID;\n";
source << "}\n";
source << "} // namespace\n";
source << "\n";
source << "uint8_t *gBinaryData = nullptr;\n";
if (readBufferSize > 0)
{
out << " gReadBuffer.resize(" << readBufferSize << ");\n";
header << "extern uint8_t gReadBuffer[" << readBufferSize << "];\n";
source << "uint8_t gReadBuffer[" << readBufferSize << "];\n";
}
for (const CallCapture &call : calls)
if (maxClientArraySize > 0)
{
out << " ";
WriteCppReplayForCall(call, &counters, out, header, &binaryData);
out << ";\n";
header << "extern uint8_t gClientArrays[" << gl::MAX_VERTEX_ATTRIBS << "]["
<< maxClientArraySize << "];\n";
source << "uint8_t gClientArrays[" << gl::MAX_VERTEX_ATTRIBS << "][" << maxClientArraySize
<< "];\n";
}
if (!binaryData.empty())
for (ResourceIDType resourceType : AllEnums<ResourceIDType>())
{
std::string dataFilepath = GetCaptureFilePath(outDir, contextId, frameIndex, ".angledata");
if (!hasResourceType[resourceType])
continue;
FILE *fp = fopen(dataFilepath.c_str(), "wb");
if (!fp)
{
FATAL() << "file " << dataFilepath << " can not be created!: " << strerror(errno);
}
fwrite(binaryData.data(), 1, binaryData.size(), fp);
fclose(fp);
std::string fname = GetCaptureFileName(contextId, frameIndex, ".angledata");
header << "std::vector<uint8_t> gBinaryData;\n";
header << "void LoadBinaryData()\n";
header << "{\n";
header << " gBinaryData.resize(" << static_cast<int>(binaryData.size()) << ");\n";
header << " FILE *fp = fopen(\"" << fname << "\", \"rb\");\n";
header << " fread(gBinaryData.data(), 1, " << static_cast<int>(binaryData.size())
<< ", fp);\n";
header << " fclose(fp);\n";
header << "}\n";
const char *name = GetResourceIDTypeName(resourceType);
header << "extern ResourceMap g" << name << "Map;\n";
source << "ResourceMap g" << name << "Map;\n";
}
else
header << "\n";
source << "\n";
source << "void ReplayContext" << contextId << "Frame(uint32_t frameIndex)\n";
source << "{\n";
source << " switch (frameIndex)\n";
source << " {\n";
for (uint32_t frameIndex = frameStart; frameIndex < frameEnd; ++frameIndex)
{
header << "// No binary data.\n";
header << "void LoadBinaryData() {}\n";
source << " case " << frameIndex << ":\n";
source << " ReplayContext" << contextId << "Frame" << frameIndex << "();\n";
source << " break;\n";
}
source << " default:\n";
source << " break;\n";
source << " }\n";
source << "}\n";
source << "\n";
source << "void LoadBinaryData(const char *fileName, size_t size)\n";
source << "{\n";
source << " if (gBinaryData != nullptr)\n";
source << " {\n";
source << " delete [] gBinaryData;\n";
source << " }\n";
source << " gBinaryData = new uint8_t[size];\n";
source << " FILE *fp = fopen(fileName, \"rb\");\n";
source << " fread(gBinaryData, 1, size, fp);\n";
source << " fclose(fp);\n";
source << "}\n";
if (maxClientArraySize > 0)
{
header
<< "void UpdateClientArrayPointer(int arrayIndex, const void *data, uint64_t size);\n";
out << "}\n";
source << "\n";
source << "void UpdateClientArrayPointer(int arrayIndex, const void *data, uint64_t size)"
<< "\n";
source << "{\n";
source << " memcpy(gClientArrays[arrayIndex], data, size);\n";
source << "}\n";
}
header << "} // anonymous namespace\n";
for (ResourceIDType resourceType : AllEnums<ResourceIDType>())
{
if (!hasResourceType[resourceType])
continue;
const char *name = GetResourceIDTypeName(resourceType);
header << "void Update" << name << "ID(GLuint id, GLsizei readBufferOffset);\n";
std::string outString = out.str();
std::string headerString = header.str();
source << "\n";
source << "void Update" << name << "ID(GLuint id, GLsizei readBufferOffset)\n";
source << "{\n";
source << " UpdateResourceMap(&g" << name << "Map, id, readBufferOffset);\n";
source << "}\n";
}
std::string cppFilePath = GetCaptureFilePath(outDir, contextId, frameIndex, ".cpp");
FILE *fp = fopen(cppFilePath.c_str(), "w");
if (!fp)
{
FATAL() << "file " << cppFilePath << " can not be created!: " << strerror(errno);
std::string headerContents = header.str();
std::stringstream headerPathStream;
headerPathStream << outDir << FmtCapturePrefix(contextId) << ".h";
std::string headerPath = headerPathStream.str();
SaveFileHelper saveHeader(headerPath);
saveHeader << headerContents;
}
fprintf(fp, "%s\n\n%s", headerString.c_str(), outString.c_str());
fclose(fp);
printf("Saved '%s'.\n", cppFilePath.c_str());
}
} // anonymous namespace
{
std::string sourceContents = source.str();
std::stringstream sourcePathStream;
sourcePathStream << outDir << FmtCapturePrefix(contextId) << ".cpp";
std::string sourcePath = sourcePathStream.str();
SaveFileHelper saveSource(sourcePath);
saveSource << sourceContents;
}
} // namespace
} // namespace
ParamCapture::ParamCapture() : type(ParamType::TGLenum), enumGroup(gl::GLenumGroup::DefaultGroup) {}
......@@ -652,7 +776,14 @@ ReplayContext::ReplayContext(size_t readBufferSizebytes,
ReplayContext::~ReplayContext() {}
FrameCapture::FrameCapture()
: mEnabled(true), mFrameIndex(0), mFrameStart(0), mFrameEnd(10), mReadBufferSize(0)
: mEnabled(true),
mClientVertexArrayMap{},
mFrameIndex(0),
mFrameStart(0),
mFrameEnd(10),
mClientArraySizes{},
mReadBufferSize(0),
mHasResourceType{}
{
reset();
......@@ -672,6 +803,12 @@ FrameCapture::FrameCapture()
mOutDirectory = pathFromEnv;
}
// Ensure the capture path ends with a slash.
if (mOutDirectory.back() != '\\' && mOutDirectory.back() != '/')
{
mOutDirectory += '/';
}
std::string startFromEnv = angle::GetEnvironmentVar(kFrameStartVarName);
if (!startFromEnv.empty())
{
......@@ -961,10 +1098,30 @@ void FrameCapture::onEndFrame(const gl::Context *context)
{
if (!mCalls.empty())
{
WriteCppReplay(mOutDirectory, context->id(), mFrameIndex, mCalls, mClientArraySizes,
mReadBufferSize);
WriteCppReplay(mOutDirectory, context->id(), mFrameIndex, mCalls);
// Count resource IDs.
for (const CallCapture &call : mCalls)
{
for (const ParamCapture &param : call.params.getParamCaptures())
{
ResourceIDType idType = GetResourceIDTypeFromParamType(param.type);
if (idType != ResourceIDType::InvalidEnum)
{
mHasResourceType[idType] = true;
}
}
}
reset();
mFrameIndex++;
// Save the index files after the last frame.
if (mFrameIndex == mFrameEnd + 1)
{
WriteCppReplayIndexFiles(mOutDirectory, context->id(), mFrameStart, mFrameEnd,
mReadBufferSize, mClientArraySizes, mHasResourceType);
}
}
}
......@@ -1022,8 +1179,10 @@ void FrameCapture::reset()
{
mCalls.clear();
mClientVertexArrayMap.fill(-1);
mClientArraySizes.fill(0);
mReadBufferSize = 0;
// Do not reset replay-specific settings like the maximum read buffer size, client array sizes,
// or the 'has seen' type map. We could refine this into per-frame and per-capture maximums if
// necessary.
}
std::ostream &operator<<(std::ostream &os, const ParamCapture &capture)
......
......@@ -162,6 +162,9 @@ class DataCounters final : angle::NonCopyable
std::map<Counter, int> mData;
};
// Used by the CPP replay to filter out unnecessary code.
using HasResourceTypeMap = angle::PackedEnumMap<ResourceIDType, bool, angle::kParamTypeCount>;
class FrameCapture final : angle::NonCopyable
{
public:
......@@ -187,6 +190,10 @@ class FrameCapture final : angle::NonCopyable
const CallCapture &call,
const ParamCapture &param);
static void ReplayCall(gl::Context *context,
ReplayContext *replayContext,
const CallCapture &call);
bool mEnabled;
std::string mOutDirectory;
std::vector<CallCapture> mCalls;
......@@ -196,10 +203,7 @@ class FrameCapture final : angle::NonCopyable
uint32_t mFrameEnd;
gl::AttribArray<size_t> mClientArraySizes;
size_t mReadBufferSize;
static void ReplayCall(gl::Context *context,
ReplayContext *replayContext,
const CallCapture &call);
HasResourceTypeMap mHasResourceType;
};
template <typename CaptureFuncT, typename... ArgsT>
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment