eval

Support PointRend, Fast_SCNN, HRNet, Deeplabv3_plus(xception, resnet, mobilenet), ContextNet, FPENet, DABNet, EdaNet, ENet, Espnetv2, RefineNet, UNet, DANet, HRNet, DFANet, HardNet, LedNet, OCNet, EncNet, DuNet, CGNet, CCNet, BiSeNet, PSPNet, ICNet, FCN, deeplab)

A small, powerful, open-source CLI coding agent that works with open models.

Track 20,710+ AI benchmark, eval, dataset, and data-quality records from 39 public sources, with linked evidence and daily updates.

minimalistic evaluator of python expression using ast module

High-performance LLM evaluation framework with parallel API calls — up to 17× faster than sequential tools. Supports box, math, and logit-based evaluation.

Serbian LLM Eval.

This course uses a deliberately vulnerable banking application to demonstrate common security vulnerabilities, their impact, and how to fix them. The application is built with Flask (backend) and React (frontend).

EvalBench is a flexible framework designed to measure the quality of generative AI (GenAI) workflows around database specific tasks.

A self-evolving Agent OS — every interaction sharpens how it judges, not just what it knows. Four ideas: self-evolution (mistakes become code gates, not logged lessons), brain-first domain brains that grow and decay, cognition that persists between sessions, and proprioception — it senses and drives its own body. Human directs. AI delivers.

Open Source Embeddings Optimisation and Eval Framework for RAG/LLM Applications. Documentations at https://docs.vectorboard.ai/introduction

A library for evaluating Retrieval-Augmented Generation (RAG) systems (The traditional ways).

Safe and fast evaluation of untrusted user-supplied python expressions

CLI for agent builders to benchmark & chaos test your AI Agents. Text, Voice, Code supported.

Automation Framework using LLM-as-a-judge to evaluate of Agentic AI, RAG, Text2SQL at scale; that is a good proxy for human judgement.

[ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games

Toolkit for measuring Claude Code and Codex performance over time against a baseline using SWEbench-lite dataset **No API key required for Max or Pro subscribers**

Multi-modal benchmark for measuring sensitive-information redaction in human recording data

多智能体应用与任务执行平台

A library that removes the __subclasses__() list from all classes, allowing for nearly absolute security in exec and eval functions.

Navi-Bench: benchmarking web agents on everyday tasks directly on real websites