An emerging AI infrastructure company scales inference from edge to cloud

Customer case study | Ceva-NeuPro Ultra

An emerging AI infrastructure company selected Ceva NeuPro technology to build a high-throughput AI inference solution for cloud deployment. The design focuses on scaling inference while addressing data-center power and total-cost pressure. 

The implementation uses a multicore NeuPro Ultra configuration for implementing a cloud inference system. 

Customer’s target markets & applications

  • AI infrastructure 
  • Cloud inference 
  • Data-center inference 
  • High-throughput, multi-user model serving 

The challenge

The customer needed high-throughput AI inference that could scale in cloud environments while containing power consumption and total cost.

Why Ceva

  • High-performance AI inference: Ceva provides the processing capabilities required for high-throughput cloud inference. 
  • Power-efficient processing at scale: The NeuPro architecture helps address the power constraints associated with scaling AI inference in data-center environments. 
  • Multicore scalability: The customer can scale inference throughput across multiple NeuPro cores within a common architectural framework. 
  • A focus on total cost: The combination of performance, power efficiency and scalability helps the customer address the total-cost pressures of cloud inference. 

Implementation Highlights

  • 8 x NeuPro Ultra (v1.6 8K version) multicore configuration 
  • Throughput scales across cores