anonymeter

TransformerLens 开源

像调试代码一样拆开GPT，定位每个注意力头的作用

A library for mechanistic interpretability of GPT-style language models

Guardrails 开源

给大模型对话装上可编程的安全护栏，防止跑偏和有害输出。

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails ···

AlignLLMHumanSurvey 开源

一张图看懂大模型如何对齐人类偏好

Aligning Large Language Models with Human: A Survey

leva 开源

用线上真实数据回放评测，让 Rails 的 LLM 改动不再靠拍脑袋

LLM Evaluation Framework for Rails apps to be used with production data.

common_metrics_on_video_quality 开源

一键计算视频生成质量指标，快速对比模型效果

You can easily calculate FVD, PSNR, SSIM, LPIPS for evaluating the quality of genera···

llm-colosseum 开源

让大模型打街霸，用对战胜负测实力

Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality ···

One-Eval 开源

用 agent 自动跑完 LLM 评测全流程，省去手工配数据与打分

Automated system for LLM evaluation via agents. Doc as below:

benchmark 开源

一套脚本横向对比多个大模型的质量

Benchmark suite for LLMs from Fireworks.ai

langfair 开源

给LLM做公平性体检，防偏见翻车

LangFair is a Python library for conducting use-case level LLM bias and fairness ass···