LLM & VLM Inference at the Edge

Large Language Models (LLMs) and Vision Language Models (VLMs) are rapidly transforming how machines understand and interact with the world. LLMs enable natural language reasoning, while VLMs combine language with image inputs to allow systems to interpret images based on textual instructions.

GSI Technology’s Gemini™-II APU enables unprecedented use cases of these workloads in edge applications.

 

Right Solution Now for Edge  Physical AI

VLMs have another super power. Their wide vector input allows other sensor inputs in addition or in place of video to be used. This results in a fast multi-modal fusion AI ability. GSI’s Gemini-II allows extremely wide input vectors, easily handling 32000-bit widths. This allows physical AI to have real world awareness of surroundings. The APU can provide time-to-first-token results from very wide vector inputs providing autonomous decision capability to a multitude of physical AI product categories.

This architectural approach enables:

  • Lower power consumption
  • Higher memory efficiency
  • Reduced system cost
  • Faster response to user requests
  • Compact deployments suitable for edge and battery-operated environments

 


Accelerating Vision Language Models

Vision Language Models require significantly more memory bandwidth than text-only models because they process images, video frames, sensor inputs, and language simultaneously.

These multimodal workloads often become constrained by memory movement rather than raw compute performance.

Gemini’s Compute-in-Memory architecture minimizes this bottleneck enabling efficient execution of workloads such as:

  • Intelligent video analytics
  • Autonomous robotics
  • Smart infrastructure
  • Industrial inspection
  • Public safety monitoring
  • Medical imaging assistance
  • Multi-camera sensor fusion
  • Defense targeting and C2 applications
  • Aerospace data processing
  • Autonomous safety awareness

Gemini has a fundamentally different architectural approach.

Instead of repeatedly transferring model weights between external memory and compute engines per pass, Gemini performs computation directly within memory in a single pass, significantly reducing data movement. This makes it particularly well suited for reasoning, scene analysis, and real-time decision making tasks based on large sets of structured and unstructured inputs.


 

©2025 GSI Technology, Inc. All Rights Reserved