Crit

Agentic toolkit for building and optimizing high-performance inference engines tailored to your GPU and desired LLM.

LLM Mart 0 views 18 listing impressions

Each engine is crit Core plus plugins, one per slot, with 33 plugins in crit's library. It uses low-bit weight projections where weights are stored as low-bit codes with one scale per group of values. Crit implements Paged KV cache with prefix sharing, where the KV cache is cut into fixed pages.

Token sampling is performed on the GPU, with the GPU applying the whole sampling law to the full vocabulary. Crit records execution choices for each phase and shape, picking eager launches, a full graph, or a partial graph from measured costs.

Summary drafted from the project's own website. Every sentence is backed by text on that page and was reviewed before publishing.

Related servers

Web search, deep research, prediction markets & crypto data for AI agents. Pay per call via x402.

21 views

Let LLM help you achieve your regression analysis with Stata

21 views

vSphere with Tanzu (VKS): Namespace and TanzuKubernetesCluster lifecycle. Requires vSphere 8.x+.

20 views

Durable agentic memory, encrypted at rest. Fully offline: no network, no API key, no cloud.

19 views