GitHub

Distributed Inference with vLLM

This repository provides reproducible recipes for deploying large language model inference at scale. Each workflow includes complete environment specifications, step-by-step instructions, and ...
IEEE

Time-delayed Interaction Model for Causal Inference between Cascading Failures in Power Grid

Abstract: The interaction model is widely used to study the propagation patterns of cascading failures in power systems. However, most existing models rely on the Markovian assumption, which limits ...
GitHub

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently ...

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor ...