A deep learning processor company has un-cloaked itself in Toronto – Tenstorrent, with a product called Grayskull.
It describes the technology behind the processor as: “The first conditional execution architecture for artificial intelligence facilitating scalable deep learning. Tenstorrent has taken an approach that dynamically eliminates unnecessary computation, thus breaking the direct link between model size growth and compute/memory bandwidth requirements.”
Conditional computation?
The Tensix cores have a packet processor, a programmable SIMD and maths computation block, five single-issue RISC cores and 1Mbyte of ram.
Associated software model
“The array of Tensix cores is stitched together with a double 2D torus network-on-chip, which facilitates multi-cast flexibility, along with minimal software burden for scheduling coarse-grain data transfers,” according to the company. “At the chip thermal design power required for a 75W bus-powered PCIE card, Grayskull achieves 368Tops and up to 23,345 sentence/second using BERT-Base for the SQuAD 1.1 data set.”
According to the Tenstorrent:
For artificial intelligence to reach the next level, machines need to go beyond pattern recognition and into cause-and-effect learning. Such machine learning models require computing infrastructure that allows them to continue growing by orders of magnitude for years to come. Machine learning computers can achieve this goal in two ways: by weakening the dependence between model size and raw compute power, through features like conditional execution and dynamic sparsity handling, and by facilitating compute scalability at hitherto unrivalled levels. Rapid changes in Machine learning models further require flexibility and programmability.
Claimed Grayskull benchmarks
- Raw TOPS – 368 TOPS
- Resnet 50, 224×224 – 22,431
- BERT base, SQUAD – 2,830sentence/s
- BERT base, SQUAD + conditional features – 10,150sentence/s
- BERT base, SQUAD + conditional features + low prec FP – 23,345sentence/s
Grayskull is aimed at inferencing in data centres, public cloud servers, private cloud servers, on-premises servers, edge servers and automotive.
Samples are said to be with partners, with the processor ready for production this autumn.
Source: electronicsweekly
