Despite being described as doing 4 x 4 matrix math, in practice, tensor core operations always seem to be working with 16 x 16 matrices, with operations being handled across two tensor cores at a time. At the end of the day there's a relatively straightforward tradeoff here between flexibility (tensor cores would be terrible at scalar operations) and throughput, as tensor cores can pack many more operations into the same die area since they are so rigid and require a fraction of the controlling logic when that cost is divided up per ALU.Ĭonsequently, while somewhat programmable, tensor cores are stuck to these types of 4 x 4 matrix multiplication-accumulation – and it’s not clear how and when the accumulation step occurs. The density has changed – they're now operating on sizable matricies instead of SIMD-packed scalar values – but the math has not. Tensor cores, while being brand-new to the GPU space, are not all that far removed from standard ALU pipelines. As it so happens, the math that tensor cores do is commonly found in deep learning training and inferencing.Īnd if this sounds familiar to a normal GPU ALU pipeline, then it should.
#Wmma 3 for mac full#
The result is a 4 x 4 FP16 or FP32 matrix NVIDIA refers to tensor cores as performing mixed precision math, because the inputted matrices are in half precision but the product can be in full precision. Tensor cores perform a fused multiply add, where two 4 x 4 FP16 matrices are multiplied and then the result added to a 4 x 4 FP16 or FP32 matrix. To recap, the tensor core is a new type of processing core that performs a type of specialized matrix math, suitable for deep learning and certain types of HPC.
Of the several mysteries around Volta’s mixed precision tensor cores, one of the more nagging ones was the capability of 4 x 4 matrix multiplication.
0 kommentar(er)
