Skip to content

S
swiftshader
Commit 1c82c7b8 by Ben Clayton
Options

Reactor (LLVM): Add support for Coroutines.

rr::Coroutines are similar to rr::Functions in that it builds a new executable function, but Coroutines have the following differences: (1) Coroutines do not support Return() statements. (2) Coroutines support Yield() statements to suspend execution of the coroutine and pass a value up to the caller. Yield can be called multiple times in a single execution of a coroutine. (3) The template argument T to Coroutine<T> is a C-style function signature. (4) Coroutine::operator() returns a rr::Stream<T> instead of an rr::Routine. (5) operator() starts execution of the coroutine immediately. (6) operator() uses the Coroutine's template function signature to ensure the argument types match the generated function signature. It is my personal opinion that (3), (5) and (6) provide a more convenient and safer interface and this could be adopted by rr::Function (post immediate milestone). Coroutines expose 3 externally callable functions: (1) 'coroutine_begin' - the main entry point of the coroutine. Allocates a coroutine stack frame, and executes up to the first call to Yield(). (2) 'coroutine_await' - the function to collect a yielded value and resume execution of the suspended coroutine. (3) 'coroutine_destroy' - the function to destruct and release the coroutine stack frame. As there are now three exposed functions for coroutines, rr::Routine::getEntry() now takes a function index. Standard rr::Functions always pass 0. rr::Nucleus::CoroutineEntries is an enumerator of these coroutine functions that correspond to the function index passed to rr::Routine::getEntry(). This change also adds third_party/llvm-7.0/stubs/Stubs.cpp. This file holds stub functions that are never called by LLVM, but are referenced by the linker. Actually linking in the file that declares these functions will drag in more referenced files, significantly slowing the build and potentially bloating the final executable size. Coroutines are not currently supported by Subzero. Bug: b/131672705 Change-Id: I0b13fe38b84c8dc85f4678019bf8d8afa7d9c37a Reviewed-on: https://swiftshader-review.googlesource.com/c/SwiftShader/+/30212Tested-by: 's avatarBen Clayton <bclayton@google.com> Presubmit-Ready: Ben Clayton <bclayton@google.com> Kokoro-Presubmit: kokoro <noreply+kokoro@google.com> Reviewed-by: 's avatarNicolas Capens <nicolascapens@google.com>
parent 45f9a930
Showing with 758 additions and 27 deletions
CMakeLists.txt
......@@ -399,6 +399,7 @@ set(LLVM_LIST
${LLVM_DIR}/lib/Analysis/MemoryDependenceAnalysis.cpp
${LLVM_DIR}/lib/Analysis/MemoryLocation.cpp
${LLVM_DIR}/lib/Analysis/MemorySSA.cpp
${LLVM_DIR}/lib/Analysis/MemorySSAUpdater.cpp
${LLVM_DIR}/lib/Analysis/MustExecute.cpp
${LLVM_DIR}/lib/Analysis/ObjCARCAliasAnalysis.cpp
${LLVM_DIR}/lib/Analysis/ObjCARCAnalysisUtils.cpp
......@@ -895,6 +896,12 @@ set(LLVM_LIST
${LLVM_DIR}/lib/Support/regstrlcpy.c
${LLVM_DIR}/lib/Target/TargetLoweringObjectFile.cpp
${LLVM_DIR}/lib/Target/TargetMachine.cpp
${LLVM_DIR}/lib/Transforms/Coroutines/CoroCleanup.cpp
${LLVM_DIR}/lib/Transforms/Coroutines/CoroEarly.cpp
${LLVM_DIR}/lib/Transforms/Coroutines/CoroElide.cpp
${LLVM_DIR}/lib/Transforms/Coroutines/CoroFrame.cpp
${LLVM_DIR}/lib/Transforms/Coroutines/CoroSplit.cpp
${LLVM_DIR}/lib/Transforms/Coroutines/Coroutines.cpp
${LLVM_DIR}/lib/Transforms/InstCombine/InstCombineAddSub.cpp
${LLVM_DIR}/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp
${LLVM_DIR}/lib/Transforms/InstCombine/InstCombineCalls.cpp
......@@ -908,9 +915,11 @@ set(LLVM_LIST
${LLVM_DIR}/lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp
${LLVM_DIR}/lib/Transforms/InstCombine/InstCombineVectorOps.cpp
${LLVM_DIR}/lib/Transforms/InstCombine/InstructionCombining.cpp
${LLVM_DIR}/lib/Transforms/IPO/BarrierNoopPass.cpp
${LLVM_DIR}/lib/Transforms/Scalar/ADCE.cpp
${LLVM_DIR}/lib/Transforms/Scalar/ConstantHoisting.cpp
${LLVM_DIR}/lib/Transforms/Scalar/DeadStoreElimination.cpp
${LLVM_DIR}/lib/Transforms/Scalar/EarlyCSE.cpp
${LLVM_DIR}/lib/Transforms/Scalar/GVN.cpp
${LLVM_DIR}/lib/Transforms/Scalar/LICM.cpp
${LLVM_DIR}/lib/Transforms/Scalar/LoopStrengthReduce.cpp
......@@ -942,6 +951,7 @@ set(LLVM_LIST
${LLVM_DIR}/lib/Transforms/Utils/SymbolRewriter.cpp
${LLVM_DIR}/lib/Transforms/Utils/VNCoercion.cpp
${LLVM_DIR}/lib/Transforms/Utils/ValueMapper.cpp
${LLVM_DIR}/../stubs/Stubs.cpp
)
if(ARCH STREQUAL "x86" OR ARCH STREQUAL "x86_64")
......
build/Visual Studio 15 2017 Win64/llvm.vcxproj
......@@ -161,6 +161,7 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\MemoryDependenceAnalysis.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\MemoryLocation.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\MemorySSA.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\MemorySSAUpdater.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\MustExecute.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\ObjCARCAliasAnalysis.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\ObjCARCAnalysisUtils.cpp" />
......@@ -434,6 +435,7 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\DebugInfo\CodeView\TypeRecordMapping.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\DebugInfo\CodeView\TypeTableCollection.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\ExecutionEngine\ExecutionEngine.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\ExecutionEngine\GDBRegistrationListener.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\ExecutionEngine\Orc\Core.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\ExecutionEngine\Orc\Legacy.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\ExecutionEngine\Orc\OrcError.cpp" />
......@@ -679,6 +681,12 @@
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Target\TargetLoweringObjectFile.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Target\TargetMachine.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroCleanup.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroEarly.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroElide.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroFrame.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroSplit.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\Coroutines.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\InstCombine\InstCombineAddSub.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\InstCombine\InstCombineAndOrXor.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\InstCombine\InstCombineCalls.cpp" />
......@@ -692,9 +700,11 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\InstCombine\InstCombineSimplifyDemanded.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\InstCombine\InstCombineVectorOps.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\InstCombine\InstructionCombining.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\IPO\BarrierNoopPass.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\ADCE.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\ConstantHoisting.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\DeadStoreElimination.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\EarlyCSE.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\GVN.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\LICM.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\LoopStrengthReduce.cpp" />
......@@ -726,6 +736,7 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Utils\SymbolRewriter.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Utils\VNCoercion.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Utils\ValueMapper.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\stubs\Stubs.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Target\X86\AsmParser\X86AsmInstrumentation.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Target\X86\AsmParser\X86AsmParser.cpp" />
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Target\X86\InstPrinter\X86ATTInstPrinter.cpp" />
......
build/Visual Studio 15 2017 Win64/llvm.vcxproj.filters
......@@ -115,6 +115,9 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\MemorySSA.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\MemorySSAUpdater.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Analysis\MustExecute.cpp">
<Filter>Source Files</Filter>
</ClCompile>
......@@ -928,6 +931,9 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\ExecutionEngine\ExecutionEngine.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\ExecutionEngine\GDBRegistrationListener.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\ExecutionEngine\Orc\Core.cpp">
<Filter>Source Files</Filter>
</ClCompile>
......@@ -1600,6 +1606,24 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Target\TargetMachine.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroCleanup.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroEarly.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroElide.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroFrame.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\CoroSplit.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Coroutines\Coroutines.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\InstCombine\InstCombineAddSub.cpp">
<Filter>Source Files</Filter>
</ClCompile>
......@@ -1639,6 +1663,9 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\InstCombine\InstructionCombining.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\IPO\BarrierNoopPass.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\ADCE.cpp">
<Filter>Source Files</Filter>
</ClCompile>
......@@ -1648,6 +1675,9 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\DeadStoreElimination.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\EarlyCSE.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Scalar\GVN.cpp">
<Filter>Source Files</Filter>
</ClCompile>
......@@ -1741,6 +1771,9 @@
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Transforms\Utils\ValueMapper.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\stubs\Stubs.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="$(SolutionDir)third_party\llvm-7.0\llvm\lib\Target\X86\AsmParser\X86AsmInstrumentation.cpp">
<Filter>Source Files</Filter>
</ClCompile>
......
src/Reactor/Coroutine.hpp 0 → 100644
// Copyright 2019 The SwiftShader Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "Reactor.hpp"
#include <memory>
#ifndef rr_ReactorCoroutine_hpp
#define rr_ReactorCoroutine_hpp
namespace rr
{
// Base class for the template Stream<T>
class StreamBase
{
protected:
StreamBase(std::shared_ptr<Routine> &routine, Nucleus::CoroutineHandle handle)
: routine(routine), handle(handle) {}
~StreamBase()
{
auto pfn = (Nucleus::CoroutineDestroy*)routine->getEntry(Nucleus::CoroutineEntryDestroy);
pfn(handle);
}
bool await(void* out)
{
auto pfn = (Nucleus::CoroutineAwait*)routine->getEntry(Nucleus::CoroutineEntryAwait);
return pfn(handle, out);
}
private:
std::shared_ptr<Routine> routine;
Nucleus::CoroutineHandle handle;
};
// Stream is the interface to a running Coroutine instance.
// A Coroutine may Yield() values of type T, which can be retrieved with
// await().
template<typename T>
class Stream : public StreamBase
{
public:
inline Stream(std::shared_ptr<Routine> &routine, Nucleus::CoroutineHandle handle)
: StreamBase(routine, handle) {}
// await() retrieves the next yielded value from the coroutine.
// Returns true if the coroutine yieled a value and out was assigned a
// new value. If await() returns false, the coroutine has finished
// execution and await() will return false for all future calls.
inline bool await(T& out) { return StreamBase::await(&out); }
};
template<typename FunctionType>
class Coroutine;
// Coroutine constructs a reactor Coroutine function.
// rr::Coroutine is similar to rr::Function in that it builds a new
// executable function, but Coroutines have the following differences:
// (1) Coroutines do not support Return() statements.
// (2) Coroutines support Yield() statements to suspend execution of the
// coroutine and pass a value up to the caller. Yield can be called
// multiple times in a single execution of a coroutine.
// (3) The template argument T to Coroutine<T> is a C-style function
// signature.
// (4) Coroutine::operator() returns a rr::Stream<T> instead of an
// rr::Routine.
// (5) operator() starts execution of the coroutine immediately.
// (6) operator() uses the Coroutine's template function signature to
// ensure the argument types match the generated function signature.
//
// Example usage:
//
// // Build the coroutine function
// Coroutine<int()> coroutine;
// {
// Yield(Int(0));
// Yield(Int(1));
// Int current = 1;
// Int next = 1;
// While (true) {
// Yield(next);
// auto tmp = current + next;
// current = next;
// next = tmp;
// }
// }
//
// // Start the execution of the coroutine.
// auto s = coroutine();
//
// // Grab the first 20 yielded values and print them.
// for (int i = 0; i < 20; i++)
// {
// int val = 0;
// s->await(val);
// printf("Fibonacci(%d): %d", i, val);
// }
//
template<typename Return, typename... Arguments>
class Coroutine<Return(Arguments...)>
{
public:
Coroutine();
template<int index>
using CArgumentType = typename std::tuple_element<index, std::tuple<Arguments...>>::type;
template<int index>
using RArgumentType = CToReactor<CArgumentType<index>>;
// Return the argument value with the given index.
template<int index>
Argument<RArgumentType<index>> Arg() const
{
Value *arg = Nucleus::getArgument(index);
return Argument<RArgumentType<index>>(arg);
}
// Completes building of the coroutine and generates the coroutine's
// executable code. After calling, no more reactor functions may be
// called without building a new rr::Function or rr::Coroutine.
// While automatically called by operator(), finalize() should be called
// as early as possible to release the global Reactor mutex lock.
inline void finalize();
// Starts execution of the coroutine and returns a unique_ptr to a
// Stream<> that exposes the await() function for obtaining yielded
// values.
std::unique_ptr<Stream<Return>> operator()(Arguments...);
protected:
std::unique_ptr<Nucleus> core;
std::shared_ptr<Routine> routine;
std::vector<Type*> arguments;
};
template<typename Return, typename... Arguments>
Coroutine<Return(Arguments...)>::Coroutine() : routine{}
{
core.reset(new Nucleus());
std::vector<Type*> types = {CToReactor<Arguments>::getType()...};
for(auto type : types)
{
if(type != Void::getType())
{
arguments.push_back(type);
}
}
Nucleus::createCoroutine(CToReactor<Return>::getType(), arguments);
}
template<typename Return, typename... Arguments>
void Coroutine<Return(Arguments...)>::finalize()
{
if(core != nullptr)
{
routine.reset(core->acquireCoroutine("coroutine", true));
core.reset(nullptr);
}
}
template<typename Return, typename... Arguments>
std::unique_ptr<Stream<Return>>
Coroutine<Return(Arguments...)>::operator()(Arguments... args)
{
finalize();
using Sig = Nucleus::CoroutineBegin<Arguments...>;
auto pfn = (Sig*)routine->getEntry(Nucleus::CoroutineEntryBegin);
auto handle = pfn(args...);
return std::unique_ptr<Stream<Return>>(new Stream<Return>(routine, handle));
}
#ifdef Yield // Defined in WinBase.h
#undef Yield
#endif
// Suspends execution of the coroutine and yields val to the caller.
// Execution of the coroutine will resume after val is retrieved.
template<typename T>
inline void Yield(const T &val) { Nucleus::yield(ValueOf(val)); }
} // namespace rr
#endif // rr_ReactorCoroutine_hpp
\ No newline at end of file
src/Reactor/LLVMReactor.cpp
......@@ -46,17 +46,20 @@
#include "llvm/IR/DataLayout.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/GlobalVariable.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/Intrinsics.h"
#include "llvm/IR/LLVMContext.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/LegacyPassManager.h"
#include "llvm/IR/LLVMContext.h"
#include "llvm/IR/Mangler.h"
#include "llvm/IR/Module.h"
#include "llvm/IR/Verifier.h"
#include "llvm/Support/Error.h"
#include "llvm/Support/TargetSelect.h"
#include "llvm/Target/TargetOptions.h"
#include "llvm/Transforms/Coroutines.h"
#include "llvm/Transforms/InstCombine/InstCombine.h"
#include "llvm/Transforms/IPO.h"
#include "llvm/Transforms/IPO/PassManagerBuilder.h"
#include "llvm/Transforms/Scalar.h"
#include "llvm/Transforms/Scalar/GVN.h"
......@@ -75,6 +78,7 @@
#include <fstream>
#include <numeric>
#include <thread>
#include <iostream>
#if defined(__i386__) || defined(__x86_64__)
#include <xmmintrin.h>
......@@ -442,6 +446,7 @@ namespace rr
const Capabilities Caps =
{
true, // CallSupported
true, // CoroutinesSupported
};
static std::memory_order atomicOrdering(llvm::AtomicOrdering memoryOrder)
......@@ -557,6 +562,10 @@ namespace rr
func_.emplace("atomic_load", reinterpret_cast<void*>(Atomic::load));
func_.emplace("atomic_store", reinterpret_cast<void*>(Atomic::store));
// FIXME (b/119409619): use an allocator here so we can control all memory allocations
func_.emplace("coroutine_alloc_frame", reinterpret_cast<void*>(malloc));
func_.emplace("coroutine_free_frame", reinterpret_cast<void*>(free));
#ifdef __APPLE__
func_.emplace("sincosf_stret", reinterpret_cast<void*>(__sincosf_stret));
#elif defined(__linux__)
......@@ -665,36 +674,47 @@ namespace rr
::module = nullptr;
}
LLVMRoutine *acquireRoutine(llvm::Function *func)
LLVMRoutine *acquireRoutine(llvm::Function **funcs, size_t count)
{
std::string name = "f" + llvm::Twine(emittedFunctionsNum++).str();
func->setName(name);
func->setLinkage(llvm::GlobalValue::ExternalLinkage);
func->setDoesNotThrow();
std::vector<std::string> mangledNames(count);
for (size_t i = 0; i < count; i++)
{
auto func = funcs[i];
std::string name = "f" + llvm::Twine(emittedFunctionsNum++).str();
func->setName(name);
func->setLinkage(llvm::GlobalValue::ExternalLinkage);
func->setDoesNotThrow();
llvm::raw_string_ostream mangledNameStream(mangledNames[i]);
llvm::Mangler::getNameWithPrefix(mangledNameStream, name, dataLayout);
}
// Compile the module - after this the llvm::Functions will have
// been freed.
std::unique_ptr<llvm::Module> mod(::module);
::module = nullptr;
mod->setDataLayout(dataLayout);
auto moduleKey = session.allocateVModule();
llvm::cantFail(compileLayer.addModule(moduleKey, std::move(mod)));
funcs = nullptr; // Now points to released memory.
std::string mangledName;
// Resolve the function addresses.
std::vector<void*> addresses(count);
for (size_t i = 0; i < count; i++)
{
llvm::raw_string_ostream mangledNameStream(mangledName);
llvm::Mangler::getNameWithPrefix(mangledNameStream, name, dataLayout);
}
llvm::JITSymbol symbol = compileLayer.findSymbolIn(moduleKey, mangledNames[i], false);
llvm::JITSymbol symbol = compileLayer.findSymbolIn(moduleKey, mangledName, false);
llvm::Expected<llvm::JITTargetAddress> expectAddr = symbol.getAddress();
if(!expectAddr)
{
return nullptr;
}
llvm::Expected<llvm::JITTargetAddress> expectAddr = symbol.getAddress();
if(!expectAddr)
{
return nullptr;
addresses[i] = reinterpret_cast<void *>(static_cast<intptr_t>(expectAddr.get()));
}
void *addr = reinterpret_cast<void *>(static_cast<intptr_t>(expectAddr.get()));
return new LLVMRoutine(addr, releaseRoutineCallback, this, moduleKey);
return new LLVMRoutine(addresses.data(), count, releaseRoutineCallback, this, moduleKey);
}
void optimize(llvm::Module *module)
......@@ -999,7 +1019,7 @@ namespace rr
::module->print(file, 0);
}
LLVMRoutine *routine = ::reactorJIT->acquireRoutine(::function);
LLVMRoutine *routine = ::reactorJIT->acquireRoutine(&::function, 1);
return routine;
}
......@@ -4168,4 +4188,317 @@ namespace rr
#endif // ENABLE_RR_DEBUG_INFO
}
} // namespace rr
// ------------------------------ Coroutines ------------------------------
namespace {
struct CoroutineState
{
llvm::Function *await = nullptr;
llvm::Function *destroy = nullptr;
llvm::Value *handle = nullptr;
llvm::Value *id = nullptr;
llvm::Value *promise = nullptr;
llvm::BasicBlock *suspendBlock = nullptr;
llvm::BasicBlock *endBlock = nullptr;
llvm::BasicBlock *destroyBlock = nullptr;
};
CoroutineState coroutine;
// Magic values retuned by llvm.coro.suspend.
// See: https://llvm.org/docs/Coroutines.html#llvm-coro-suspend-intrinsic
enum SuspendAction
{
SuspendActionSuspend = -1,
SuspendActionResume = 0,
SuspendActionDestroy = 1
};
} // anonymous namespace
namespace rr {
void Nucleus::createCoroutine(Type *YieldType, std::vector<Type*> &Params)
{
// Types
auto voidTy = ::llvm::Type::getVoidTy(*::context);
auto i1Ty = ::llvm::Type::getInt1Ty(*::context);
auto i8Ty = ::llvm::Type::getInt8Ty(*::context);
auto i32Ty = ::llvm::Type::getInt32Ty(*::context);
auto i8PtrTy = ::llvm::Type::getInt8PtrTy(*::context);
auto promiseTy = T(YieldType);
auto promisePtrTy = promiseTy->getPointerTo();
auto handleTy = i8PtrTy;
auto boolTy = i1Ty;
// LLVM intrinsics
auto coro_id = ::llvm::Intrinsic::getDeclaration(::module, llvm::Intrinsic::coro_id);
auto coro_size = ::llvm::Intrinsic::getDeclaration(::module, llvm::Intrinsic::coro_size, {i32Ty});
auto coro_begin = ::llvm::Intrinsic::getDeclaration(::module, llvm::Intrinsic::coro_begin);
auto coro_resume = ::llvm::Intrinsic::getDeclaration(::module, ::llvm::Intrinsic::coro_resume);
auto coro_end = ::llvm::Intrinsic::getDeclaration(::module, ::llvm::Intrinsic::coro_end);
auto coro_free = ::llvm::Intrinsic::getDeclaration(::module, ::llvm::Intrinsic::coro_free);
auto coro_destroy = ::llvm::Intrinsic::getDeclaration(::module, ::llvm::Intrinsic::coro_destroy);
auto coro_promise = ::llvm::Intrinsic::getDeclaration(::module, ::llvm::Intrinsic::coro_promise);
auto coro_done = ::llvm::Intrinsic::getDeclaration(::module, ::llvm::Intrinsic::coro_done);
auto coro_suspend = ::llvm::Intrinsic::getDeclaration(::module, ::llvm::Intrinsic::coro_suspend);
auto allocFrameTy = ::llvm::FunctionType::get(i8PtrTy, {i32Ty}, false);
auto allocFrame = ::module->getOrInsertFunction("coroutine_alloc_frame", allocFrameTy);
auto freeFrameTy = ::llvm::FunctionType::get(voidTy, {i8PtrTy}, false);
auto freeFrame = ::module->getOrInsertFunction("coroutine_free_frame", freeFrameTy);
// Build the coroutine_await() function:
//
// bool coroutine_await(CoroutineHandle* handle, YieldType* out)
// {
// if (llvm.coro.done(handle))
// {
// return false;
// }
// else
// {
// *value = (T*)llvm.coro.promise(handle);
// llvm.coro.resume(handle);
// return true;
// }
// }
//
llvm::FunctionType *coroutineAwaitTy = llvm::FunctionType::get(boolTy, {handleTy, promisePtrTy}, false);
::coroutine.await = llvm::Function::Create(coroutineAwaitTy, llvm::GlobalValue::InternalLinkage, "coroutine_await", ::module);
::coroutine.await->setCallingConv(llvm::CallingConv::C);
{
auto args = ::coroutine.await->arg_begin();
auto handle = args++;
auto outPtr = args++;
::builder->SetInsertPoint(llvm::BasicBlock::Create(*::context, "co_await", ::coroutine.await));
auto doneBlock = llvm::BasicBlock::Create(*::context, "done", ::coroutine.await);
auto resumeBlock = llvm::BasicBlock::Create(*::context, "resume", ::coroutine.await);
auto done = ::builder->CreateCall(coro_done, {handle}, "done");
::builder->CreateCondBr(done, doneBlock, resumeBlock);
::builder->SetInsertPoint(doneBlock);
::builder->CreateRet(::llvm::ConstantInt::getFalse(i1Ty));
::builder->SetInsertPoint(resumeBlock);
auto promiseAlignment = ::llvm::ConstantInt::get(i32Ty, 4); // TODO: Get correct alignment.
auto promisePtr = ::builder->CreateCall(coro_promise, {handle, promiseAlignment, ::llvm::ConstantInt::get(i1Ty, 0)});
auto promise = ::builder->CreateLoad(::builder->CreatePointerCast(promisePtr, promisePtrTy));
::builder->CreateStore(promise, outPtr);
::builder->CreateCall(coro_resume, {handle});
::builder->CreateRet(::llvm::ConstantInt::getTrue(i1Ty));
}
// Build the coroutine_destroy() function:
//
// void coroutine_destroy(CoroutineHandle* handle)
// {
// llvm.coro.destroy(handle);
// }
//
llvm::FunctionType *coroutineDestroyTy = llvm::FunctionType::get(voidTy, handleTy, false);
::coroutine.destroy = llvm::Function::Create(coroutineDestroyTy, llvm::GlobalValue::InternalLinkage, "coroutine_destroy", ::module);
::coroutine.destroy->setCallingConv(llvm::CallingConv::C);
{
auto handle = ::coroutine.destroy->arg_begin();
::builder->SetInsertPoint(llvm::BasicBlock::Create(*::context, "", ::coroutine.destroy));
::builder->CreateCall(coro_destroy, {handle});
::builder->CreateRetVoid();
}
// Begin building the main coroutine_begin() function.
//
// CoroutineHandle* coroutine_begin(<Arguments>)
// {
// YieldType promise;
// auto id = llvm.coro.id(0, &promise, nullptr, nullptr);
// void* frame = coroutine_alloc_frame(llvm.coro.size.i32());
// CoroutineHandle *handle = llvm.coro.begin(id, frame);
//
// ... <REACTOR CODE> ...
//
// end:
// SuspendAction action = llvm.coro.suspend(none, true /* final */); // <-- RESUME POINT
// switch (action)
// {
// case SuspendActionResume:
// UNREACHABLE(); // Illegal to resume after final suspend.
// case SuspendActionDestroy:
// goto destroy;
// default: // (SuspendActionSuspend)
// goto suspend;
// }
//
// destroy:
// coroutine_free_frame(llvm.coro.free(id, handle));
// goto suspend;
//
// suspend:
// llvm.coro.end(handle, false);
// return handle;
// }
//
llvm::FunctionType *functionType = llvm::FunctionType::get(handleTy, T(Params), false);
::function = llvm::Function::Create(functionType, llvm::GlobalValue::InternalLinkage, "coroutine_begin", ::module);
::function->setCallingConv(llvm::CallingConv::C);
#ifdef ENABLE_RR_DEBUG_INFO
::debugInfo = std::unique_ptr<DebugInfo>(new DebugInfo(::builder, ::context, ::module, ::function));
#endif // ENABLE_RR_DEBUG_INFO
auto entryBlock = llvm::BasicBlock::Create(*::context, "coroutine", ::function);
::coroutine.suspendBlock = llvm::BasicBlock::Create(*::context, "suspend", ::function);
::coroutine.endBlock = llvm::BasicBlock::Create(*::context, "end", ::function);
::coroutine.destroyBlock = llvm::BasicBlock::Create(*::context, "destroy", ::function);
::builder->SetInsertPoint(entryBlock);
Variable::materializeAll();
::coroutine.promise = ::builder->CreateAlloca(T(YieldType), nullptr, "promise");
::coroutine.id = ::builder->CreateCall(coro_id, {
::llvm::ConstantInt::get(i32Ty, 0),
::builder->CreatePointerCast(::coroutine.promise, i8PtrTy),
::llvm::ConstantPointerNull::get(i8PtrTy),
::llvm::ConstantPointerNull::get(i8PtrTy),
});
auto size = ::builder->CreateCall(coro_size, {});
auto frame = ::builder->CreateCall(allocFrame, {size});
::coroutine.handle = ::builder->CreateCall(coro_begin, {::coroutine.id, frame});
// Build the suspend block
::builder->SetInsertPoint(::coroutine.suspendBlock);
::builder->CreateCall(coro_end, {::coroutine.handle, ::llvm::ConstantInt::get(i1Ty, 0)});
::builder->CreateRet(::coroutine.handle);
// Build the end block
::builder->SetInsertPoint(::coroutine.endBlock);
auto action = ::builder->CreateCall(coro_suspend, {
::llvm::ConstantTokenNone::get(*::context),
::llvm::ConstantInt::get(i1Ty, 1), // final: true
});
auto switch_ = ::builder->CreateSwitch(action, ::coroutine.suspendBlock, 3);
// switch_->addCase(::llvm::ConstantInt::get(i8Ty, SuspendActionResume), trapBlock); // TODO: Trap attempting to resume after final suspend
switch_->addCase(::llvm::ConstantInt::get(i8Ty, SuspendActionDestroy), ::coroutine.destroyBlock);
// Build the destroy block
::builder->SetInsertPoint(::coroutine.destroyBlock);
auto memory = ::builder->CreateCall(coro_free, {::coroutine.id, ::coroutine.handle});
::builder->CreateCall(freeFrame, {memory});
::builder->CreateBr(::coroutine.suspendBlock);
// Switch back to the entry block for reactor codegen.
::builder->SetInsertPoint(entryBlock);
#if defined(_WIN32)
// FIXME(capn):
// On Windows, stack memory is committed in increments of 4 kB pages, with the last page
// having a trap which allows the OS to grow the stack. For functions with a stack frame
// larger than 4 kB this can cause an issue when a variable is accessed beyond the guard
// page. Therefore the compiler emits a call to __chkstk in the function prolog to probe
// the stack and ensure all pages have been committed. This is currently broken in LLVM
// JIT, but we can prevent emitting the stack probe call:
::function->addFnAttr("stack-probe-size", "1048576");
#endif
}
void Nucleus::yield(Value* val)
{
ASSERT_MSG(::coroutine.id != nullptr, "yield() can only be called when building a Coroutine");
// promise = val;
//
// auto action = llvm.coro.suspend(none, false /* final */); // <-- RESUME POINT
// switch (action)
// {
// case SuspendActionResume:
// goto resume;
// case SuspendActionDestroy:
// goto destroy;
// default: // (SuspendActionSuspend)
// goto suspend;
// }
// resume:
//
RR_DEBUG_INFO_UPDATE_LOC();
Variable::materializeAll();
// Types
auto i1Ty = ::llvm::Type::getInt1Ty(*::context);
auto i8Ty = ::llvm::Type::getInt8Ty(*::context);
// Intrinsics
auto coro_suspend = ::llvm::Intrinsic::getDeclaration(::module, ::llvm::Intrinsic::coro_suspend);
// Create a block to resume execution.
auto resumeBlock = llvm::BasicBlock::Create(*::context, "resume", ::function);
// Store the promise (yield value)
::builder->CreateStore(V(val), ::coroutine.promise);
auto action = ::builder->CreateCall(coro_suspend, {
::llvm::ConstantTokenNone::get(*::context),
::llvm::ConstantInt::get(i1Ty, 0), // final: true
});
auto switch_ = ::builder->CreateSwitch(action, ::coroutine.suspendBlock, 3);
switch_->addCase(::llvm::ConstantInt::get(i8Ty, SuspendActionResume), resumeBlock);
switch_->addCase(::llvm::ConstantInt::get(i8Ty, SuspendActionDestroy), ::coroutine.destroyBlock);
// Continue building in the resume block.
::builder->SetInsertPoint(resumeBlock);
}
Routine* Nucleus::acquireCoroutine(const char *name, bool runOptimizations)
{
ASSERT_MSG(::coroutine.id != nullptr, "acquireCoroutine() called without a call to createCoroutine()");
::builder->CreateBr(::coroutine.endBlock);
#ifdef ENABLE_RR_DEBUG_INFO
if (debugInfo != nullptr)
{
debugInfo->Finalize();
}
#endif // ENABLE_RR_DEBUG_INFO
if(false)
{
std::error_code error;
llvm::raw_fd_ostream file(std::string(name) + "-llvm-dump-unopt.txt", error);
::module->print(file, 0);
}
// Run manadory coroutine transforms.
llvm::legacy::PassManager pm;
pm.add(llvm::createCoroEarlyPass());
pm.add(llvm::createCoroSplitPass());
pm.add(llvm::createCoroElidePass());
pm.add(llvm::createBarrierNoopPass());
pm.add(llvm::createCoroCleanupPass());
pm.run(*::module);
if(runOptimizations)
{
optimize();
}
if(false)
{
std::error_code error;
llvm::raw_fd_ostream file(std::string(name) + "-llvm-dump-opt.txt", error);
::module->print(file, 0);
}
llvm::Function *funcs[Nucleus::CoroutineEntryCount];
funcs[Nucleus::CoroutineEntryBegin] = ::function;
funcs[Nucleus::CoroutineEntryAwait] = ::coroutine.await;
funcs[Nucleus::CoroutineEntryDestroy] = ::coroutine.destroy;
Routine *routine = ::reactorJIT->acquireRoutine(funcs, Nucleus::CoroutineEntryCount);
::coroutine = CoroutineState{};
return routine;
}
} // namespace rr
\ No newline at end of file
src/Reactor/LLVMRoutine.hpp
......@@ -18,6 +18,8 @@
#include "Routine.hpp"
#include <cstdint>
#include <stddef.h>
#include <vector>
namespace rr
{
......@@ -26,20 +28,22 @@ namespace rr
class LLVMRoutine : public Routine
{
public:
LLVMRoutine(void *ent, void (*callback)(LLVMReactorJIT *, uint64_t),
LLVMRoutine(void **entries, size_t entry_count,
void (*callback)(LLVMReactorJIT *, uint64_t),
LLVMReactorJIT *jit, uint64_t key)
: entry(ent), dtor(callback), reactorJIT(jit), moduleKey(key)
: entries(entries, entries+entry_count),
dtor(callback), reactorJIT(jit), moduleKey(key)
{ }
virtual ~LLVMRoutine();
const void *getEntry()
const void *getEntry(int index)
{
return entry;
return entries[index];
}
private:
const void *entry;
const std::vector<const void *> entries;
void (*dtor)(LLVMReactorJIT *, uint64_t);
LLVMReactorJIT *reactorJIT;
......
src/Reactor/Nucleus.hpp
......@@ -64,6 +64,26 @@ namespace rr
static void createFunction(Type *ReturnType, std::vector<Type*> &Params);
static Value *getArgument(unsigned int index);
// Coroutines
using CoroutineHandle = void*;
template <typename... ARGS>
using CoroutineBegin = CoroutineHandle(ARGS...);
using CoroutineAwait = bool(CoroutineHandle, void* yieldValue);
using CoroutineDestroy = void(CoroutineHandle);
enum CoroutineEntries
{
CoroutineEntryBegin = 0,
CoroutineEntryAwait,
CoroutineEntryDestroy,
CoroutineEntryCount
};
static void createCoroutine(Type *ReturnType, std::vector<Type*> &Params);
Routine *acquireCoroutine(const char *name, bool runOptimizations = true);
static void yield(Value*);
// Terminators
static void createRetVoid();
static void createRet(Value *V);
......
src/Reactor/Reactor.hpp
......@@ -58,7 +58,8 @@ namespace rr
{
struct Capabilities
{
bool CallSupported; // Support for rr::Call()
bool CallSupported; // Support for rr::Call()
bool CoroutinesSupported; // Support for rr::Coroutine<F>
};
extern const Capabilities Caps;
......
src/Reactor/Reactor.vcxproj
......@@ -306,6 +306,7 @@
<ClInclude Include="Reactor.hpp" />
<ClInclude Include="Routine.hpp" />
<ClInclude Include="Thread.hpp" />
<ClInclude Include="Traits.hpp" />
<ClInclude Include="x86.hpp" />
</ItemGroup>
<ItemGroup>
......
src/Reactor/Reactor.vcxproj.filters
......@@ -80,5 +80,8 @@
<ClInclude Include="Thread.hpp">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="Traits.hpp">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
</Project>
\ No newline at end of file
src/Reactor/ReactorUnitTests.cpp
......@@ -13,6 +13,7 @@
// limitations under the License.
#include "Reactor.hpp"
#include "Coroutine.hpp"
#include "gtest/gtest.h"
......@@ -1369,6 +1370,82 @@ TYPED_TEST(GEPTest, PtrOffsets)
delete routine;
}
TEST(ReactorUnitTests, Coroutines_Fibonacci)
{
if (!rr::Caps.CoroutinesSupported)
{
SUCCEED() << "Coroutines not supported";
return;
}
Coroutine<int()> function;
{
Yield(Int(0));
Yield(Int(1));
Int current = 1;
Int next = 1;
While (true) {
Yield(next);
auto tmp = current + next;
current = next;
next = tmp;
}
}
auto coroutine = function();
int32_t expected[] =
{
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597,
2584, 4181, 6765, 10946, 17711, 28657, 46368, 75025, 121393, 196418,
317811,
};
auto count = sizeof(expected) / sizeof(expected[0]);
for (size_t i = 0; i < count; i++)
{
int out = 0;
EXPECT_EQ(coroutine->await(out), true);
EXPECT_EQ(out, expected[i]);
}
}
TEST(ReactorUnitTests, Coroutines_Parameters)
{
if (!rr::Caps.CoroutinesSupported)
{
SUCCEED() << "Coroutines not supported";
return;
}
Coroutine<uint8_t(uint8_t* data, int count)> function;
{
Pointer<Byte> data = function.Arg<0>();
Int count = function.Arg<1>();
For(Int i = 0, i < count, i++)
{
Yield(data[i]);
}
}
uint8_t data[] = {10, 20, 30};
auto coroutine = function(&data[0], 3);
uint8_t out = 0;
EXPECT_EQ(coroutine->await(out), true);
EXPECT_EQ(out, 10); out = 0;
EXPECT_EQ(coroutine->await(out), true);
EXPECT_EQ(out, 20); out = 0;
EXPECT_EQ(coroutine->await(out), true);
EXPECT_EQ(out, 30); out = 99;
EXPECT_EQ(coroutine->await(out), false);
EXPECT_EQ(out, 99);
EXPECT_EQ(coroutine->await(out), false);
EXPECT_EQ(out, 99);
}
int main(int argc, char **argv)
{
::testing::InitGoogleTest(&argc, argv);
......
src/Reactor/Routine.hpp
......@@ -24,7 +24,7 @@ namespace rr
virtual ~Routine();
virtual const void *getEntry() = 0;
virtual const void *getEntry(int index = 0) = 0;
// Reference counting
void bind();
......
src/Reactor/SubzeroReactor.cpp
......@@ -136,6 +136,7 @@ namespace rr
const Capabilities Caps =
{
false, // CallSupported
false, // CoroutinesSupported
};
enum EmulatedType
......@@ -497,8 +498,9 @@ namespace rr
void seek(uint64_t Off) override { position = Off; }
const void *getEntry() override
const void *getEntry(int index) override
{
ASSERT(index == 0); // Subzero does not support multiple entry points per routine yet.
if(!entry)
{
position = std::numeric_limits<std::size_t>::max(); // Can't stream more data after this
......@@ -3479,4 +3481,9 @@ namespace rr
void EmitDebugLocation() {}
void EmitDebugVariable(Value* value) {}
void FlushDebug() {}
void Nucleus::createCoroutine(Type *YieldType, std::vector<Type*> &Params) { UNIMPLEMENTED("createCoroutine"); }
Routine* Nucleus::acquireCoroutine(const char *name, bool runOptimizations) { UNIMPLEMENTED("acquireCoroutine"); return nullptr; }
void Nucleus::yield(Value* val) { UNIMPLEMENTED("Yield"); }
}
third_party/llvm-7.0/stubs/Stubs.cpp 0 → 100644
// Copyright 2019 The SwiftShader Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// This file contains LLVM stub functions - these are functions that are never
// called and are declared to satisfy the linker.
#include <assert.h>
#define STUB() assert(!"Stub function. Should not be called");
#include "llvm/Transforms/IPO/PassManagerBuilder.h"
namespace llvm
{
void PassManagerBuilder::addExtension(ExtensionPointTy Ty, ExtensionFn Fn) { STUB(); }
} // namespace llvm
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment