I think you are quite a bit confused here. John clearly says the optimization passes are done at the SPIR-V -> LLVM level. It doesn’t matter if the GLSL code is optimized or not.
Talos Principle developer says that their performance issues may be related to shaders but they don’t have a definite proof, and I don’t see how it is related to the issue you posted on Github.
Which compiler are they using?
I don’t know, but it seems that Vulkan applications are shipping SPIR-V shaders. It is then compiled and optimized to intermediate LLVM bytecode and ultimately in assembly code by the driver. Therefore it is up to the driver to perform shader optimization passes. I have to check that up.
The Valve video does seem to touch on that shader compiling. And other issues they ran into that hurt performance until they got them fixed, which could be affecting Talos Principle.
Once they’re compiled for the particular drivers and cached, shouldn’t there not be an issue?
Obviously it might be a bit of a pain, and Valve talks about matching hardware and drivers to deliver the compiled shaders instead of doing it each time the game is run for each person.
[quote=“libcg, post:76, topic:3174”]
The problem is that your post is implying that it is a major issue with the Vulkan API when it is an early bug that will be fixed quick[/quote]
Bug ? There’s absolutely no bug. They only implemented a reference shader compiler that does no optimizations at all. That’s no bug, that’s by design, and that’s the root of the issue for many developers.
If you guys look at the Vulkan SDK, as @INovaeFlavien mentioned, all that it comes with is a reference compiler. If you look at the source code to this compiler you will notice that it more or less doesn’t do any optimization whatsoever. If you had actually read the link I posted to the GitHub issue you would know that Khronos has an LLVM based SPIR-V compiler that only works with OpenCL. LunarG, the guys who made the Vulkan SDK, wrote another LLVM SPIR-V compiler however it only outputs GLSL. Thus, while it is possible to get optimized Vulkan shaders, you have to take the following steps: GLSL shader -> LunarG Ref Compiler -> SPIR-V -> LunarGLASS -> Optimized GLSL -> LunarG Ref Compiler (again) -> final SPIR-V.
The Talos Principle seems to have just used the reference compiler however it’s entirely possible that Epic and Valve went through the entire chain of events that I outlined above to get optimized SPIR-V. It’s also possible that they are using different compilers that are either proprietary or that I’m currently unaware of. As for Epic’s Android demo the Android SDK is managed by Google and for all I know maybe they have a separate compiler?
Either way this is a serious cluster fuck compared to D3D12’s simple D3DCompile() which allows me to pass flags that control the optimization level and voila, I have DXBC that I can now hand off to the GPU.
NVIDIA is shipping a compiler in the form of an extension. So, you can submit GLSL instead of SPIR-V.
This would 1) have the same problems as OpenGL (poorly optimized shaders) and 2) only work on NVIDIA hardware
I was wondering whether #2 could be mitigated by submitting GLSL and then asking for the raw SPIR-V back. I haven’t finished reading the Vulkan spec. Can SPIR-V be read back from a shader? Nothing can be done about #1 though.
The LLVM path was supposed to be ideal as it would draw on all the optimization passes that LLVM offers, but that did not seem to pan out. 
Unless there’s a separate extension to get SPIR-V it won’t give you SPIR-V. The driver compiler is compiling it into ASM for the target hardware, at no point is it getting converted into SPIR-V.
This brings up another question: is GLSL a good language? I know how to use it, but I’ve never worked in one of those 20,000 line shaders I hear so much about. So, I don’t have much of an opinion of it. I ask because if a ton of effort is about to be dumped into fixing the shader compilation ecosystem, maybe it would be a good opportunity to build out a new language spec entirely. I know this isn’t an immediate solution to your problem, but I’m just curious to what your take is.
I prefer HLSL over GLSL but I’ve heard that Metal’s shading language is really well done - I haven’t worked with it yet myself.
What I can tell from experience is that 95% of the shader code in GLSL is similar ( to the point of just requiring some simple copy/paste ) to HLSL.
The remaining 5% is mostly the declarations ( textures, samplers, constant buffers, layouts… ). But it’s a huge pain. I think HLSL is better designed in that area than GLSL but it can be pretty subjective. In any case, if you’re porting your shaders from one to the other, that’s probably where most of your efforts will go to.
Where are the Vulkan API docs, anyway? Is there not a website that lists all the functions, their arguments, and what they do? I looked and couldn’t find it.
Looking at Valve’s video, I’m under the impression they have many different forms of shader compiling that’s dependent on the hardware and drivers. If there was just a reference compiler, wouldn’t they just be compiled once by the developer generically? Unless I’m misunderstanding something.
Anyway, ultimately in the end, with DirectX11 you are limiting it to Windows, and you may run into bottlenecks for things you want to do in the future (larger battles, more dense ring systems, especially worse performance on highly threaded CPUs)
DirectX12, obviously that’s Win10 only.
With Vulkan, lets say it is 35% slower in the worst case scenario, well it’s probably still going to be playable for most people. It’ll still probably be smooth. It won’t bottleneck getting more optimizations outside of the shader fill rate.
Not even hoping that these problems are resolved so that it’s better than DirectX11 in every way, it still seems like it’d be the clear choice.
It’d be nice to hope all these problems are solved. I think they will be, since so many developers all over are adopting it, but even if they aren’t it seems like the better choice to me.
https://www.khronos.org/registry/vulkan/
I’m confident the issues will be solved at some point it’s just a question of when. If we add support for D3D12 in the near-term instead of Vulkan it’s likely we’d still add Vulkan support at some point in the future - which is why we’d prefer to just support Vulkan now however, as mentioned above, a ~30%+ perf hit unless we jump through a bunch of shader compilation hoops is pretty serious. We’ll see how things play out over the next few months before we make a final decision.
Lets say, two months from now a couple “small” optimizations made it in and that ~30% got dropped in half to 15%; what then?
Hard to say, would depend on a number of variables such as Win10 adoption rate and whether or not that performance fix accompanies a clear roadmap for closing the gap on the remaining 15% with a proper compiler.
But if you start with DirectX12, you’re leaving out a lot of supporters.
Wouldn’t it make more sense to start with Vulkan, if the issues turn out to not get solved, to then have a DirectX12 render option?
Seems like it’d make more sense to scrap work using the current pipeline(DirectX11, right?) and use Vulkan.
Also, after all, isn’t much of how the DirectX12 API works similar to Mantle? Like isn’t it the biggest rework of the API in over a decade while DirectX7-11 have all been incremental steps?
I was under the impression that AMD worked with both and both were based on Mantle’s way of doing things that gave huge performance increases in Battlefield4, for example.
So I imagine whatever rendering pipeline you set up for Vulkan, you’ll have a much more drop-in replacement than D3D11>Vulkan or D3D11>D3D12 is compared to Vulkan>D3D12. And Vulkan runs on the same hardware DirectX11 does, so you don’t really need DirectX11 support to begin with except for a lot of extra work for a bit of potential performance gains if those issues aren’t worked out. But as far as work and hardware coverage involved…
Obviously you know more than me there on the work involved, but that’s how it’d seem to me. It seems to me that you would be able to streamline development more supporting Vulkan and DX12, than using DX11 and DX12 or all 3.
It seems to me that lots of people like Windows7 and don’t want to upgrade.
I mean look, right now for a year, Windows10 is FREE for Windows7 and Windows8 legal customers. It’s not like cost is keeping them from upgrading. Lots of people just don’t want it.
?!? 
You sold us DX11 game.
Give me one example of a game project that gained more success by being exclusive to some platform.
Game looks fabulous now.
Next game sure, knock yourself out.
By the time Battlescape is released most of us will have better specs.
So the lower performance is only your wild guess for the future.
My wild guess is that Vulkan will mature.
After all, Valve is pushing this thing for some time… …hmm gee i wonder why?
And Valve is not some indie studio… that does not make em infallible, no. But still.
I say lets not jump on any bandwagons and get back to work.
This can be done after the fact, if needed.
Yes it would be harder, but still doable.
Remember that you wanted to make game first and gain experience by doing so ?
So you can one day build that ultimate Space sim ?
Sorry guys, but YOU ARE DOING IT AGAIN! Stop perfecting engine. It is GOOD ENOUGH.
Have to agree with Dirk here. Always looking at the next great shiny will never make a computer game. Not unless you have serious resources.