Yes they do seem to think I have misjudged and, while that's certainly possible, after reading their commentary I believe it's far more likely that they are the ones missing the point. bridgman even admits his "compiler-fu is pretty weak" so please allow me to explain. There are 2 levels of optimization with Vulkan/D3D: source -> IR/IL and IR/IL -> ISA. Vulkan's drivers being in beta would affect IR/IL -> ISA which can absolutely have an impact on performance. I am, however, less concerned about this because if you look at D3D11 vs OpenGL this was never the primary problem.
OpenGL frequently has slower shaders because it only performs source -> ISA compilation which you can clearly see in the Talos benchmarks. Due to the fact that this has to be performed during runtime it's very time sensitive and thus the driver does not spend a lot of time optimizing. This is the key advantage of D3D, its HLSL compiler, and the DXBC IL. With D3D you can compile your shaders as a preprocess before the game ships which allows the compiler to spend much more time optimizing. During runtime the driver then only needs to take optimized DXBC and compile it to the ISA.
Thus the process of going from IR/IL/DXBC to ISA was never the primary bottleneck. The problem is compiling source code to an optimized IR/IL or, in the case of OpenGL, the fact that it skips that step entirely. Vulkan's shader compiler ecosystem (currently) being a complete disaster is because there's (currently) no easy way to optimize your SPIR-V. At the moment you have to go through GLSL -> glslang -> SPIR-V -> LunarGLASS -> GLSL -> glslang -> SPIR-V to get an optimized shader. Is it possible? Yes. Is it a major pain in the ass? Absolutely. Presumably the Talos guys did not go through this process and instead just did glslang -> SPIR-V which, if you look at its source code, you will see does little to no optimization whatsoever.
The belief that only the driver should do shader optimization is a holdover from OpenGL group-think which benchmarks consistently show is completely ridiculous. There seems to be a lot of people quickly forming the opinion that I don't know what I'm talking about just because I'm slamming Vulkan however they haven't actually bothered to do their homework. For starters I'm not slamming, or even talking about, the Vulkan API. I'm only talking about its shader compilers which the API is dependent on. Quit forming emotional opinions based on drinking the open source Kool-Aid, learn how D3D/Vulkan/OGL actually work, and go look at the compiler source code if you don't believe my criticisms are accurate. After all isn't that the whole point of open source?