This is a fork of the original lm-sys/FastChat repo, but with support for evaluating the MT-Bench scores of language models in 6 languages (en, ru, ja, zh, de, fr, in, vi, pl). See here for more ...

This benchmark suite measures mutex performance under different contention patterns by spawning multiple threads that repeatedly acquire and release a shared lock. The key aspects of our methodology: ...